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This  publication  is  primarily  a  working  paper.  It  Is  published  solely  to  docuaent  work  performed 


swtwtv 


This  paper  reviews  the  research  literature  on  teaching  effectiveness  In  the  public  school 
classroom  and  suggests  ways  In  which  this  literature  could  be  useful  In  enhancing  current  Air 
Training  Command  (ATC)  Instructor  evaluation  procedures.  Specifically,  this  paper  identifies  key 
Indicators  of  teaching  effectiveness  froa  the  literature,  compares  these  Indicators  with  those 
currently  used  on  ATC  For*  281  and  by  the  A nay  and  Navy,  Identifies  several  observational  and 
measurement  formats  suitable  for  feedback  enhancement,  and  provides  conclusions  and 
recommendations  concerning  the  feasibility  of  a  new  ATC  feedback  enhancement  initiative. 
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PREFACE 


This  technical  paper  represents  the  Initial  feasibility  phase  In  a  possibly 
longer-term  effort  to  develop  a  feedback  enhancement  procedure  for  Air  Training  Cotaund 
(ATC)  Instructors  that  would  co«R>lement  current  ATC  Fore  281  and  related  evaluation 
procedures.  Should  a  decision  favorable  to  developing  an  observation/measurement 
procedure  for  enhancing  feedback  to  ATC  Instructors  follow  from  this  paper,  the 
development  and  pilot-testing  of  a  proposed  feedback  system  would  constitute  additional 
phases  of  the  longer-term  activity. 
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AIR  FORCE  INSTRUCTOR  EVALUATION  ENHANCEMENT: 
EFFECTIVE  TEACHING  BEHAVIORS  AND  ASSESSMENT  PROCEDURES 


I.  INTRODUCTION 

The  purpose  of  this  technical  paper  Is  to  Investigate  the  feasibility  of  Implementing  an 
evaluation  and  feedback  system  for  enhancing  the  teaching  effectiveness  of  Air  Training  Counand 
(ATC)  Instructors.  Its  primary  purpose  Is  to  review  state-of-the-art  literature  on  Indicators  of 
teaching  effectiveness,  compare  those  Indicators  to  those  currently  In  use  In  the  public  school 
sector  and  on  ATC  Form  281,  Instructor  Evaluation  Checklist,  and  to  prioritize  several 
measurement  and/or  observation  procedures  suitable  for  providing  feedback  to  the  ATC  Instructor. 
The  work  detailed  In  this  paper  Is  based  on  the  assumption  that  the  teaching  effectiveness 
Indicators  and  measurement  procedures  described  herein  would  be  used  to  complement  and  enhance 
the  Form  281  currently  In  use  by  ATC. 

Accordingly,  this  document  has  been  organized  Into  the  following  additional  sections: 

II.  State-of-the-art  Indicators  of  teaching  effectiveness  derived  from  the  research 
1  Iterature.  This  section  reviews  the  research  conducted  In  public  school  classrooms  from  1960  to 
the  present  and  Indicates  those  key  Instructor  behaviors  and  procedures  which  have  been  found  to 
Increase  learners'  performance  on  cognitive  tests  of  academic  achievement. 

III.  Instructor  evaluation  procedures  In  the  public  school  sector.  This  section  reviews 
some  of  the  evaluation  and  feedback  systems  currently  In  use  by  school  systems  and  State 
Departments  of  Education  to  evaluate  and  provide  feedback  to  public  school  teachers.  The 
teaching  effectiveness  Indicators  used  In  one  of  these  systems  are  compared  with  those  from  the 
research  literature. 

IV.  Description  of  Instructor  evaluation  forms  In  the  Air  Force,  Army,  and  Navy  and  how 
they  are  used  In  their  Instructor  environments.  This  section  summarizes  the  current  evaluation 
procedure  used  by  ATC  and  compares  the  teaching  effectiveness  Indicators  on  Form  281  with  those 

reported  In  the  research  literature  and  those  currently  In  use  by  the  Army  and  Navy. 

V.  Survey  of  observation  and  measurement  formats  for  feedback  enhancement.  This  section 
reviews  several  practical  means  by  which  the  Indicators  of  teaching  effectiveness  presented  In 
Section  II  could  be  recorded  and  used  In  a  professional  development  and/or  self-improvement 

feedback  system  for  ATC  Instructors.  A  prioritization  of  formats  and  Individuals  available  to 
serve  as  data  recorders /observers  Is  provided  based  on  estimates  of  acceptability  to  users,  ease 
of  Implementation,  cost,  objectiveness,  and  Increases  In  feedback,  validity,  and  reliability. 

VI.  Summary  and  reco— endatlons.  This  section  simmtarlzes  the  main  findings  of  this  paper 
and  discusses  the  Implications  and  possible  impact  of  an  enhanced  feedback  system  for  Increasing 
ATC  Instructor  performance  and  the  Impact  of  such  a  system  In  other  sectors  of  the  ATC,  such  as 
training  In  the  ATC  Instructor's  and  supervisor's  course,  revision  of  ATC  Form  281,  and 

development  of  a  peer  evaluation  form,  a  flel d/laboratory-based  version  of  Form  281,  and  a  new 
student  evaluation  form. 


II.  STATE-OF-THE-ART  INDICATORS  OF  TEACHING  EFFECTIVENESS 
OERIVED  FROM  THE  RESEARCH  LITERATURE 

From  the  research  literature  dating  from  1960  to  the  present,  approximately  10  teacher 
behaviors  have  shown  promising  empirical  relationships  to  desirable  student  outcomes,  primarily 
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Measured  as  achievement  on  classroom  and  standardized  tests  of  academic  performance.  These  10 
behaviors  have  com  fro*  studies  at  both  the  eleMntary  and  secondary  school  levels  that  have 
used  classroom  Interaction  analysis  (Evertson  &  Green,  1986;  Rosenshlne  &  Furst,  1973)  to 
directly  observe  sequences  of  student-teacher  behavior  (dialogue)  In  the  classroom  which 
correlate  with  student  performance  on  cognitive  tests  of  academic  achievement.  Five  behaviors 
resulting  from  these  studies  have  strong  research  support  while  another  five  have  promising 
support  and  also  appear  logically  related  to  effective  teaching.  The  first  five  behaviors  will 
be  Identified  as  "key"  behaviors,  since  for  purposes  of  this  review  these  will  be  considered 
essential  for  effective  teaching.  The  second  five  will  be  Identified  as  "catalytic"  or  "helping" 
behaviors  which  may  occur  In  various  mixtures  to  assist  In  ImpleMnting  the  key  behaviors.  In 
this  section  an  overview  of  these  five  key  behaviors  will  be  provided  first  and,  then,  the 
subdlMnslons  or  Individual  components  of  each  of  the  five  will  be  described.  The  five  key 
teaching  behaviors  for  which  empirical  research  studies  conducted  In  public  school  classrooms 
have  provided  strong  support  are: 


1 .  Cl  arl  ty 

2.  Variety 

3.  Task  orientation 

4.  EngageMnt  in  the  learning  process 

5.  Moderate  to  high  rates  of  success 


Clarity 


This  behavior  refers  to  how  clear  and  Interpretable  an  Instructor's  presentation  Is  to  the 
class.  For  example,  are  the  points  the  Instructor  Is  making  understandable?  Is  the  Instructor 
able  to  explain  concepts  clearly  such  that  the  students  are  able  to  follow  in  a  logical 

step-by-step  order?  And,  Is  the  Instructor's  oral  delivery  to  the  class  clear,  audible  and 
Intelligible,  and  free  of  any  distracting  mannerisms  that  could  Impair  a  student's  understanding 
of  the  content.  One  result  f- on  the  research  on  teacher  clarity  Is  that  Instructors  have  been 
found  to  vary  considerably  on  this  behavior.  That  Is,  not  all  Instructors  are  able  to 

communicate  clearly  and  directly  to  their  students  without  wandering,  without  speaking  above  the 
"heads"  of  students  or  without  speech  patterns  (for  example,  speaking  too  f;.-t)  that  Impair  the 
clarity  of  what  Is  being  presented. 

The  research  has  Indicated  that  Instructors  who  teach  with  high  degrees  of  clarity  generally 
spend  less  time  having  to  go  over  material  and  generally  have  their  questions  answered  correctly 
the  first  time  through  the  material,  allowing  more  time  for  Instruction.  Clarity  Is  a  complex 

behavior,  since  It  Is  related  to  maqy  other  so-called  cognitive  behaviors,  such  as  an 

instructor's  organization  of  the  content,  familiarity  with  the  lesson,  and  delivery  strategies 
(e.g.,  whether  an  Instructor  uses  the  proper  mix  of  discussion,  question  and  answer,  and  lecture 
at  the  appropriate  tlMs).  Nevertheless,  both  the  cognitive  clarity  and  oral  clarity  of  an 
Instructor's  presentation  have  been  found  to  vary  substantially  among  teachers  which.  In  turn, 
has  produced  significant  differences  In  student  performance  on  cognitive  tests  of  achievement. 


Variety 

This  behavior  refers  to  variability  or  flexibility  of  delivery  during  the  presentation  of  a 
lesson.  It  Includes,  for  example,  the  planned  mixing  of  different  classroom  behaviors,  such  as 
lecturing,  group  discussion,  question  and  answers,  and  Independent  practice.  For  example.  If, 
across  several  related  lessons,  an  Instructor’s  lesson  format  utilized  these  and  other 
Instructional  techniques  to  weave  a  single,  unified  theme  or  achieve  a  predetermined  behavioral 
objective,  the  Instructor  could  be  said  to  be  using  a  high  degree  of  variety.  Research  has 
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Indicated  that  the  use  of  variety  In  Instructional  aaterlals  and  techniques.  In  the  types  of 
reinforcements  used,  and  In  the  types  of  feedback  given  to  students  Is  related  to  Increases  In 
student  achievement. 

One  of  the  most  popular  and  effective  ways  of  creating  variety  during  Instruction  Is  to  ask 
varying  types  of  questions.  Many  different  types  of  questions  have  been  Identified  (Sail  et  al., 
1978)  and,  when  Integrated  Into  the  pacing  and  sequencing  of  a  lesson,  have  provided  a  means  of 
creating  meaningful  variation  within  a  lesson.  An  Instructor's  knowledge  of  question-asking 
behavior  and  the  ability  to  discriminate  and  use  the  many  different  types  of  question  formats 
(e.g.,  fact  questions,  process  questions,  convergent  questions,  divergent  questions)  In  the 
process  of  delivering  a  lesson  have  been  found  to  correlate  with  student  achievement.  These  and 
other  types  of  questioning  formats  have  been  the  theme  of  numerous  texts  and  training  materials 
(e.g.,  Cunningham,  1971;  Hunklns,  1976;  Survey,  1974)  and  continue  to  be  a  popular  focus  of 
classroom  research. 

Another  aspect  of  variety  In  teaching  Is  the  use  of  learning  materials,  equipment,  displays, 
and  space  In  the  classroom.  This  Includes  the  visual  variety  of  the  classroom  and  the  extent  to 
which  this  variety  can  actually  encourage  student  Involvement  with  lesson  content.  The  display 
of  graphic  materials,  use  of  audiovisual  devices,  display  of  maps  and  globes,  and  organization  of 
different  reference  materials  have  been  considered  by  students  and  Instructors  to  contribute  to 
Instructional  variety,  which.  In  turn,  has  been  found  to  Influence  student  behavior  during 
Instruction  and  performance  on  end-of-unlt  tests.  In  some  studies  the  amount  of  attention  In  a 
classroom  was  greater  for  classrooms  which  used  greater  amounts  of  variety  In  classroom 
activities  and  materials;  other  studies  have  noted  changes  related  to  student  test  performance. 


Task  Orientation 


This  behavior  refers  to  the  degree  to  which  the  Instructor  Is  achievement-oriented  and  has 
high  expectations  with  respect  to  his  or  her  students.  Task-related  aspects  of  this  behavior  are 
the  amount  of  time  the  Instructor  spends  lecturing  and  asking  questions,  encouraging  students  to 
Inquire  or  think  Independently,  and  the  amount  of  Intellectual  or  cognitive  emphasis  provided  by 
the  Instructor.  These  aspects  of  task  orientation  often  manifest  themselves  In  the  degree  of 
concern  the  Instructor  has  that  all  relevant  material  gets  covered  and  learned,  as  opposed  to  a 
preoccupation  with  procedural  matters  or  an  exclusive  concern  that  the  students  simply  enjoy  the 
class.  This  behavior  also  can  be  found  In  the  extent  to  which  the  Instructor  coawunlcates  high 
performance  expectations  to  the  class  early  in  the  course  of  Instruction.  Although  researchers 
agree  that  the  classrooms  should  provide  a  friendly  and  considerate  context  In  which  to  learn, 
most  agree  that  achievement  has  been  higher  In  classrooms  that  espouse  primarily  a  business-1  ike 
atmosphere  and  high  performance  expectations.  That  Is,  classrooms  In  which  Instructor-student 
Interactions  focus  more  on  Intellectual  content  and  performance  than  on  process  Issues  (such  as 
how  to  use  materials  or  classroom  rules  and  procedures)  are  more  likely  to  have  higher  rates  of 
achievement  than  those  that  do  not.  It  has  also  been  rroorted  that  the  Instructors  in 
content-oriented  classrooms  were  more  conversant  with  the  type  of  content  likely  to  appear  on 
department  and  standardized  achievement  tests.  An  instructor  with  a  high  degree  of  task 
orientation  would  be  one  who  Is  goal -oriented— that  Is,  who  knows  what  behavioral  outcomes  he  or 
she  wants  to  achieve  In  a  given  period  of  time,  organizes  Instruction  around  these  observable 
outcomes,  and  adheres  steadfastly  to  them  in  the  midst  of  other  less  Instructional^  relevant 
activities  which  may  arise  from  time  to  time  to  create  distractions.  The  systematic  organization 
of  class  content,  the -use  of— this  organization  to  present  performance-oriented  lessons  using 
teaching  strategies  that  emphasize  high  performance  expectations  all  have  been  found  to  be 
Important  Ingredients  of  an  Instructor's  task  orientation. 


3 


Engagement  tn  the  Learning  Process 

As  noted  above,  an  Instructor  who  Is  task-oriented  Is  likely  to  maximize  the  amount  of  time  a 
student  has  to  learn  the  material  to  be  tested.  However,  a  distinctly  different  concept  than  the 
efficient  use  of  Instructional  time  Is  the  time  students  are  actually  engaged  In  learning  the 
material.  This  has  been  called  the  engagement  rate  of  students,  or  the  on-task  behavior  of 
students. 

Engagement  rate  Is  the  percentage  of  time  devoted  to  learning  In  which  the  student  Is 
actually  on-task  and  engaged  with  the  Instructional  materials  and  activities  being  presented. 
The  key  to  understanding  engagement  rate  Is  the  awareness  that  while  an  Instructor  can  be 
task-oriented,  providing  maximum  content  coverage,  the  students  may  not  be  engaged  or  on-task 
during  some  or  all  of  this  time.  This  disengagement  can  Involve  an  emotional  or  mental 
detachment  from  the  1esson—or  both.  Off-task  or  nonengaged  behavior  can  take  both  obvious  and 
nonobvlous  forms.  When  students  are  physically  Inattentive  (e.g.,  looking  out  the  window  of 
preoccupied  with  off-task  materials),  they  obviously  are  not  engaged  In  Instruction,  however 
thoroughly  It  Is  being  presented  by  the  Instructor.  However,  research  has  shown  that  students 
can  also  be  nonengaged  In  far  more  subtle  ways,  as  with  a  student  who  looks  attentive  but  whose 
thoughts  are  “miles*  away.  Correcting  this  type  of  nonengagement  may  be  a  far  more  difficult 
task.  Several  research  studies  have  contributed  useful  data  for  Increasing  learning  time  and, 
most  Importantly,  the  student  engagement  rate.  From  these  data  It  was  possible  for  Crawford  et 
al.  (1978)  to  Identify  behaviors  that  have  potential  for  Increasing  learning  time  that  results  In 
greater  amounts  of  on-task  behavior.  Some  of  their  suggestions  are  the  following: 

1.  Instructors  should  have  a  system  of  rules  that  coMunlcate  to  students  what  Is  expected 
and  how  learning  will  proceed  In  their  classrooms  prior  to  the  start  of  the  Instruction  (Brophy  A 
Evertson,  1976;  Stallings  A  Kaskowltz,  1974). 

2.  Instructors  should  frequently  move  around  the  room  to  monitor  students'  work  and 
communicate  to  the  students  an  awareness  of  their  progress  (McDonald  A  Ellas,  1976;  Stallings  A 
Kaskowltz,  1974). 

3.  Mhen  students  work  Independently,  Instructors  should  ensure  that  the  assignments  are 
challenging  but  can  be  completed  by  each  student  working  without  Instructor  direction  (McDonald  A 
Ellas,  1976;  Stallings  A  Kaskowltz,  1974). 

4.  Instructors  should  minimize  such  activities  as  giving  directions  and  organizing  the 
class  for  Instruction,  by  writing  the  dally  schedule  on  the  board  when  appropriate,  ensuring  that 
students  know  how  to  proceed  and  what  to  do  without  asking  or  taking  class  time  (McDonald  A 
Ellas,  1976;  Soar  A  Soar,  1973). 

5.  Instructors  should  make  abundant  use  of  textbooks,  workbooks,  and  other  paper-and-pencll 
activities  which  are  at  or  slightly  above  a  student's  current  level  of  functioning  (Brophy  A 
Evertson,  1976;  McDonald  A  Ellas,  1976;  Stallings  A  Kaskowltz,  1974). 

6.  Instructors  should  avoid  “timing  errors*;  that  Is,  they  should  prevent  Inattentiveness 
by  making  speedy  transitions  from  one  activity  to  another  (Brophy  A  Evertson,  1976). 


Moderatm  to  High  Success  Rat» 

A  crucial  aspect  of  research  that  has  Investigated  task  orientation  and  student  engagement 
has  been  the  level  of  difficulty  at  which  material  Is  presented.  Level  of  difficulty  In  these 
studies  was  measured  as  the  rate  at  which  students  could  understand  and  correctly  complete 
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exercises  pertaining  to  the  material  being  taught.  One  possible  level  of  difficulty  Is  that  of 
"high  success,"  In  which  the  student  understands  the  task  and  makes  only  occasional  careless 
errors;  another  Is  "medium  success,"  1r.  which  the  student  has  partial  understanding,  but  makes 
some  substantive  errors;  and  a  third  Is  "low  success,"  In  which  the  student  does  not  understand 
the  task  at  all.  The  findings  for  task  orientation  and  student  engagement  reported  above  are 
highly  related  to  level  of  difficulty— or  success  rate.  The  findings  consistently  point  out  that 
Instruction  that  produces  a  moderate  to  high  success  rate  results  In  Increased  achievement.  In 
addition,  research  has  shown  that  Instruction  producing  low  error  rates  can  contribute  to  high 
levels  of  student  self-esteem  and  to  positive  attitudes  toward  the  subject  matter.  The  average 
student  In  a  typical  public  school  classroom  spends  about  half  of  the  time  working  on  tasks  that 
provide  the  opportunity  for  high  success.  Researchers  have  found  that  students  who  spend  more 
than  half  their  time  In  high  success  activities  had  higher  achievement  scores,  better  retention, 
and  more  positive  attitudes  toward  their  class.  These  findings  have  led  to  at  least  one 
suggestion  that  students  spend  from  60-701  of  thalr  time  on  tasks  that  afford  the  opportunity  for 
moderate  to  high  levels  of  success  (Brophy  t  Evertson,  1976);  that  Is,  tasks  that  allow  for 
almost  complete  understanding  and  only  occasional  careiess  errors  the  first  time  through  the 
material. 

Moderate  to  high  success  rates  tend  to  produce  mastery  of  the  lesson  content  which  also 
Increases  the  likelihood  that  the  student  will  be  able  to  apply  learned  knowledge  at  a  later  time 
In  some  practical  way  (e.g.,  solving  problems,  making  decisions,  choosing  correct  solutions). 
Exercises  providing  moderate  to  high  success  rates  allow  the  Individual  elements  or  pieces  that 
are  1  earned  to  "fall  Into  place,"  thereby  providing  responses  that  are  smooth  and  automatic. 
Research  has  Indicated  that  many  Instructors  devote  Insufficient  time  to  this  stage  of  learning 
(Fisher  et  al.,  1978)  which  Is  believed  to  be  particularly  crucial  for  mastery  of  the  content  and 
the  generalization  of  the  content  learned  to  the  on-the-job  training  environment.  Organizing  and 
planning  Instruction  that  yields  moderate  to  high  success  rates  but  at  the  same  time  Is  not 
boring,  repetitive  or  time-wasting  are  considered  to  be  a  key  behavior  for  effective  teaching. 


Simmwry  of  Kay  Behaviors 

The  five  concepts  of  clarity,  variety,  task  orientation,  student  engagement  rate,  and  success 
rate  are  considered  to  be  among  the  most  Important  general  teaching  behaviors  comprising  modern 
definitions  of  effective  teaching.  It  Is  generally  reported  that  without  the  knowledge  and  skill 
to  present  lessons  that  are  clear,  that  Incorporate  variety,  that  are  task-oriented,  and  that 
actually  engage  students  In  the  learning  process  at  moderate  to  high  rates  of  success,  an 
Instructor  would  not  be  effective  In  producing  desirable  patterns  of  student  performance.  In  the 
remaining  portions  of  this  section  we  will  present  the  subcomponents  of  each  of  these  five 
behaviors  and  Identify  some  observable  signs  by  which  they  can  be  measured  and  feedback  provided 
to  the  ATC  Instructor.  In  addition,  the  following  discussion  will  Identify  five  helping  or 
catalytic  behaviors  generally  considered  useful  In  Implementing  the  five  key  behaviors.  These 
helping  behaviors  are  the  Instructor's  use  of  structuring,  questioning,  probing,  student  Ideas, 
and  enthusiasm. 


Clarity  In  the  Classroom 

Table  1  sunwarfzes  some  of  the  subdimensions  or  specific  components  by  which  current  research 
has  defined  the  concept  of  Instructional  clarity.  The  first  two  columns  In  Table  1  list  some 
indicators  of  clarity  likely  to  be  observed  among  effective  and  Ineffective  Instructors.  The 
third  column  suggests  some  of  the  types  cf  feedback  that  might  be  given  to  an  '  ’structor  who  Is 
observed  exhibiting  one  or  more  Ineffective  Indices  of  clarity. 
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Table  1.  Effective  end  Ineffective  Indicators  for  Clarity 


.C  </» 


a>  vi  *3 

•'  5  S3 

—  * 

41 

3k  +4  +*  -C 

♦r  e  <o  m 

**  "c  VI  "S  c 

41  O  ^ 

*—  •*-  4*  C 

tv)  C  <p  T3 

c  fc-  a> 

a»  «o  c  vi 

U  JSz  41  ©  3 

4»  »■—  wl 

4-  C  VI  41 

° 

4)  O  4)  *— 

>  O  -C  ■*- 

41  -  d-»  d->  X 

*”  S»  «  !•  1. 

■a  *a  •*-»  o  o 

2  41  <  ^ 

*—  O  d-»  > 

-*•  1  -r-  U  4 

VI  5  XI  m  f 

«  5  3  5  2 


k  41  O  *— 

3  c  f-  I 

U  O  41 

3  U  >  +* 

£  41  C 

* «  g’-S 

w  b  VI  V  e 

r  ^  vi  «  o 

i. 

«  2  ♦*  4 

ll  "li 

*  5  s  •  *  tS 

5  v  i  •  s 

5 

<  «  O  O  4» 

w  It  V  y 


Of  d-» 

VI  *  c 

VI  S.  4» 

41  VI  o  i  e 

4>  C  c 

W  49*0  010 

3  Kans 

U  d->  u  VI  VI  41 

u  c  vjr- 

4  #  0*0  41  41 

«  V*  4)  u  J5 

>—  C  VI  *  f  a  d-> 

y  ^  l  .  d3 

4)  Of  3  V 

?  S  £  *  5  -  ° 

5  z  z  ..  *  §  r 

£  8  3  3  £  £ 

41  Co>H- 

O  V  *  3  « 

c  «*-  *r>  f  O' 

5  3  «  5  £  8 


*o  f* 
c  c  c 
L  «  L 
«  a 
41  C  41 
*—  O  r- 

VI 

L  VI  L 
O  41  O 


>  U  > 
41  O  41 
*«•  <4- 

41  41 

I-  *0  U. 
I  41  I 
«  k  dd 
VI  T-  vi 

3  §-3 

<m  £  d-> 


4)  •*  • 

pl  f  x 
«  e  4i 
£  O  -M 

3  6^0 

3  O  d-»  <J 
4-  C 
c  <y  u 

l.  ■p  V  JJ 

o  c  p> 

«*-  >.  o  w 

49  O  <Q 

I  §  5  z 

41  VI  «*  «*- 
VI  Id  £ 
2|  «  v  dd 

a  f 

£  ^  **  ° 
d-»  k.  +4 
C  OJ  i» 

o  c  e  <a 
vi  t.  a. 
VI  O  4 
41  4)  41  V) 


♦4  Ol  VI 

C  c 
4)  f  O 

V  g  V 

§  3  +> 
u  £  S 

oiv  a 
C  C  v*  e 
f  f  4>  d-» 
♦f  C  H 
e  4i 

41  VI  £  ta 

VI  fc.  d-»  c 

£•**  ^  o 

<*»  2  o 

44  44  L. 

2  I  X  4! 

t  i  * 

ass£ 

co  X  v>  j3 


*  ••  ■—  a 

V.  *»  VI 
VI  01  <J  VI 
U  N  i  «i 

8  C  S-- 

t  <«  u  4> 

*  f  at| 

e  3 
v»  ^  f  <k 

*  8  c  ^ 

•*•  c  o  o 

>  4  VI  N 

2>  vi  *0 
'?  u  e 
a.  -tj 


o 

4=  ^Oe 

S 

*  VI  «  » 

v  v  c 

^Ci 

01  «•  VI  41 

*»  ^  Z- 

§  J!  9  , 

u  **  -s  i 

X  V. 

5  o  i.  « 
S  o-  o  i 

O  Ol  VI 

**  5  s.  c 

M  Jl  «  « 

S  Sj  sa  3> 

e  £  o  u 


•—  ■—  o» 

Of  VI  £ 

>  ■*•  u 

4!  8-  S 

v.  2 

S  2  2 

i  ■—  a  -o 

f  o.  5 

a  °  v. 

.-S 

s  * 

•  O  U  e 

•  e  c  >, 
oi  m  o  C 

•  44  o  *> 
41  VI  VI 

l>  k  VI 

C  4  °  S 
O  C  VI  $ 
VI  3  44  C 
VI  u 
4>  <4.  «  <4. 

f  O  <k  -*• 


VI  3 

Is 

-  -f  g 

VI  o.  4C 

Uvi 
4  -  -  9 

4)  C  41  O 
o  f  >  u 

ef  o 

VI  Ol 


4»  o>  e 

N  I  4  f 

C  Ol  L  d) 
«  v  4|  5 

2*  S  ?  v, 


o  o 

i;2 

41  O 

f  e»  e 

u55 

y  ^  5 


8  *  = 

♦4  »-  O 

VI  ^  3  • 

♦i  fc*  a  >i 
e  4i  i  i- 

«  Ol  C  dd 
VI  m  O  U 

£C  4-  •*- 
2  C  3 
Q-  «  o* 


6 


Knows  ability  levels  and  teaches  Falls  to  know  that  Instruction  Deteralne  ability  level  froe 

at  or  slightly  above  learners'  Is  under  or  over  heads  of  tests,  previous  asslgnaents 

current  level  functioning;  e.g.,  students.  Seees  not  to  know  and  Interests  and  retarget 

knows  students'  attention  spans.  when  aost  learners  have  "tuned  Instruction  accordingly. 

out." 


Being  deer  Poor  clarity 

(An  effective  instructor! _ (An  ineffective  Instructor)  _ (Reco— ended  behavior) _ 

Uses  exaaples,  illustrations  and  Restricts  presentation  to  Restate  aain  points  in  at  least  one 
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The  research  evidence.  The  primary  and  secondary  sources  which  have  been  consulted  which 
provide  strong  or  premising  research  evidence  for  the  subdimensions  of  clarity  presented  In  Table 
1  are  as  follows:1 


Primary 

Alexander,  L.,  Franklewuz,  R.,  A  Williams,  R.  (1979).  Facilitation  of  learning  and  retention  of 
oral  Instruction  using  advance  and  post  organizers.  Journal  of  Educational  Psychology,  71, 
701  -707. 

Ausubel,  D.P.  (1960).  The  use  of  advanced  organizers  In  the  learning  and  retention  of  meaningful 
material.  Journal  of  Educational  Psychology,  51 ,  267-272. 

Blaney,  R.  (1983).  Effects  of  teacher  structuring  and  reacting  on  achievement.  Elementary  School 
Journal ,  83,  569-577. 

Brophy,  J.E.,  A  Evertson,  C.M.  (1976).  Learning  from  teaching:  A  developmental  perspective. 
Boston:  Allyn  and  Bacon. 

Clark,  C.,  Gage,  N.,  Marx,  R. ,  Peterson,  P. ,  Strayrouk,  N.,  A  Winnie,  P.  (1979).  A  factorial 

experiment  on  teacher  structuring,  soliciting  and  reacting.  Journal  of  Educational 
Psychology,  71  ,  534-552. 

Evertson,  C. ,  Emmer,  E.T. ,  A  Brophy,  J.E.  (1980).  Predictors  of  effective  teaching  In  junior 
high  mathematics  classrooms.  Journal  of  Research  In  Mathematics  Education,  11,  169-178. 

Good,  T.L.,  A  Grouws,  D.A.  (1979).  The  Missouri  mathematics  effectiveness  project.  Journal  of 
Educational  Psychology,  71.  355-362, 

Hiller,  J.H.,  Fisher,  G.,  A  Kaess,  W.  (1969).  A  computer  Investigation  of  verbal  characteristics 
of  effective  classroom  lecturing.  American  Educational  Research  Journal,  6,  161-675. 

Kennedy,  J.,  Crulckshank,  D.,  Bush,  A.,  A  Myers,  B.  (1978).  Additional  Investigations  into  the 
nature  of  clarity.  Journal  of  Educational  Research,  72,  3-10. 

Land,  M.  (1979).  Low-Inference  variables  and  teacher  clarity:  Effects  on  student  concept 
learning.  Journal  of  Educational  Psychology.  71.  795-799. 

Land,  M.,  A  Sarith,  L.  (1979).  Effect  of  a  teacher  clarity  variable  on  student  achievement. 

Journal  of  Educational  Research,  72,  196-197. 

Land,  M.,  A  Sarith,  L.  (1979).  The  effect  of  low  Inference  teacher  clarity  Inhibitors  on  student 
achievement.  Journal  of  Teacher  Education,  30,  55-57. 

Smith,  L. ,  A  Cotten,  M.  (1980).  Effect  of  lesson  vagueness  and  discontinuity  on  student 
achievement  and  attitudes.  Journal  of  Educational  Psychology,  72,  670-675. 

Smith,  L. ,  A  Land,  M.  (1981).  Low  Inference  verbal  behaviors  related  to  teacher  clarity. 

Journal  of  Classroom  Interaction,  1  7,  37-42. 


1  "Primary"  refers  to  an  original  research  study.  "Secondary"  refers  to  a  review  of 
research  or  meta  analysis  which  summarizes  the  results  of  more  than  a  single  study. 
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Smith,  L. ,  A  Sanders,  K.  (1981).  The  effects  on  student  achievement  and  student  perception  of 
varying  structure  In  social  studies  content.  Journal  of  Educational  Research,  74,  333-336. 


Secondary 

Belgard,  M.,  Rosenshlne,  6.,  A  Gage,  N.L.  (1971).  Effectiveness  In  explaining:  Evidence  on  Its 
generality  and  correlation  with  pupil  ratings.  In  Ian  Westbury  A  A. A.  Bellack  (Eds.), 
Research  Into  classroom  processes:  Recent  developments  and  next  steps.  New  York:  Teachers 
College  Press. 

Brophy,  J.,  A  Good,  T.L.  (1986).  Teacher  behavior  and  student  achievement.  In  M.C.  wittrock 
(Ed.),  Handbook  of  research  on  teaching  (3rd  ed.,  pp.  328-374).  New  York:  Macmillan. 

Cohen,  P.A.,  Ebellng,  B.J.,  A  Kullk,  J.A.  (1981).  A  meta-analysis  of  outcome  studies  of  visual  - 
based  Instruction.  Education  Conwunl  cation  and  Technology  Journal,  29,  26-36. 

Dunkln,  M.J.,  A  Biddle,  B.J.  (1974).  The  study  of  teaching.  New  York:  Holt. 

Lulten,  J.,  Ames,  W.,  A  Aerson  (1980).  A  meta-analysis  of  advanced  organizers  on  learning  and 
retention.  American  Educational  Research  Journal,  17,  211-218. 

Melton,  R.  (1978).  Resolution  of  conflicting  claims  concerning  the  effect  of  behavioral 
objectives  on  student  learning.  Review  of  Educational  Research,  48,  291-302. 

Rosenshlne,  B.  (1971).  Teaching  behaviors  and  student  achievement.  London:  National  Foundation 
for  Educational  Research  In  England  and  Wales. 

Rosenshlne,  B.  (1968).  To  explain:  A  review  of  research.  Educational  Leadership.  26,  275-280. 

Rationale  for  the  behaviors  comprising  teacher  clarity.  The  following  section  presents  an 
overview  of  the  rationale  for  each  of  the  subcomponents  of  clarity  based  on  the  above-cited 
research  literature. 

As  Is  noted  In  Table  1,  the  ability  to  be  "clear"  Involves  seven  important  behaviors.  Three 
of  these  pertain  to  behaviors  that  should  occur  at  the  beginning  of  a  lesson,  whereas  the  other 
four  pertain  to  behaviors  that  are  Implemented  during  a  lesson.  The  three  behaviors  that  should 
occur  at  the  beginning  of  a  lesson  are: 

1.  Informing  the  learner  of  the  objective. 

2.  Providing  learners  with  an  advanced  organizer. 

3.  Checking  for  task-relevant  prior  learning  and  reteaching.  If  necessary. 

Being  clear  means  being  understood  by  learners.  However,  being  understood  Is  believed  to 
depend  as  much  on  what  an  Instructor  does  prior  to  teaching  a  lesson  as  what  Is  done  during  It. 
Therefore,  these  three  "pre-lnstructlonal"  activities  are  recocneended  to  establish  a  learning  set 
which  focuses  learners  In  the  Intended  direction. 

Typically,  at  the  start  of  a  lesson,  learners  do  not  know  what  behaviors,  skills  or  concepts 
will  be  expected  of  them.  This  Is  considered  to  be  an  unintentional  source  of  anxiety,  since  In 
the  absence  of  Information,  unrealistic  expectations  may  be  created  In  the  minds  of  learners 
which  are  anxiety-producing.  Anxiety,  and  especially  high  levels  of  anxiety,  has  been  reported 
to  shorten  learners'  attention  spans  and  provide  a  timely  reason  for  not  "tuning-in"  the 
instruction.  Informing  learners  of  the  objective  conveys  the  expectations  the  Instructor  has  for 
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them  and  dispels  any  unrealistic  fears  and  expectations  that  eight  have  been  created.  If 
learners  know  what  to  expect  and.  If  the  expectation  Is  reasonable  In  their  eyes,  a  desirable 
learning  set  Is  established. 

Advanced  organizers  at  the  start  of  the  lesson  are  often  believed  necessary  to  keep  learners 
tuned-in  and  on-task.  Advanced  organizers  place  the  day’s  learning  In  perspective  by  connecting 
It  with  prior  and  future  parts  of  the  curriculum.  With  advanced  organizers  a  lesson  Is  more 
likely  to  be  seen  by  learners  as  Important,  as  part  of  something  larger  than  the  lesson  Itself 
and,  therefore,  as  more  valued  as  part  of  this  larger  context.  If  the  content  to  be  taught 
appears  Irrelevant  or  unattached  to  anything  that  has  gone  before  or  Is  likely  to  come  after,  It 
often  becomes  content  for  which  learners  believe  they  will  not  be  held  accountable. 

A  third  pre-lnstructlonal  activity  for  achieving  clarity  Involves  checking  for  task-relevant 
prior  learning.  Without  the  mastery  of  task-relevant  prior  learning,  a  lesson  may  be 
Incomprehensible  to  some.  If  not  most,  learners.  A  lesson  Is  thought  to  have  clarity  only  to  the 
extent  that  all  or  the  majority  of  students  can  exhibit  the  prerequisite  facts,  skills  or 
concepts  required  for  understanding  the  lesson.  To  achieve  this.  Inquiries  are  made  before  or  at 
the  beginning  of  a  lesson  to  see  If  learners  can  actually  exhibit  the  prerequisite  behaviors  for 
new  learning  to  occur.  Many  options  are  suggested  In  the  research  literature  for  conducting  such 
an  Inquiry,  from  Informal  oral  questions  asked  randomly  of  a  few  students  to  a  more  formal 
checking  of  homework  or  assignments.  When  extensive  deficiencies  In  task-relevant  prior 
knowledge  are  found,  reteaching  old  but  prerequisite  content  is  considered  more  Important  than 
teaching  new  content  scheduled  for  the  day's  lesson.  Some  techniques  coomionly  recommended  for 
checking  task-relevant  prior  learning  Include: 

1.  Preparing  on  Index  cards,  before  the  class  begins,  a  pool  of  questions  which  some 
students  will  be  asked  to  randomly  choose  from  and  orally  answer  at  the  start  of  class. 

2.  Asking  a  single  brief  question  on  the  main  point  taught  In  the  previous  lesson. 

3.  Calling  on  students  to  suamrlze  one  major  Idea  of  their  own  choosing  from  a  previous, 
related  lesson. 

4.  Asking  students  to  apply  a  specific  rule,  regulation  or  process  that  was  taught  In  the 
previous  lesson. 

5.  Having  students  write  out  the  answers  to  several  key  questions  placed  on  the  board,  while 
the  Instructor  circulates  and  checks  answers. 

6.  Having  students  write  the  answers  to  several  key  questions  and  check  them  with  a 
classmate. 

7.  Having  students  prepare  questions  about  the  previous  lesson  and  ask  them  of  each  other. 

The  four  behaviors  that  should  occur  during  a  lesson  In  order  to  achieve  clarity  are: 

1.  Giving  directives  slowly  and  distinctly. 

2.  Knowing  the  ability  levels  of  the  learners  and  teaching  at  or  slightly  above  their 
current  level  of  functioning. 

3.  Using  examples.  Illustrations  and  demonstrations  to  explain  and  clarify  text  and  workbook 
content. 


4.  Providing  a  review  or  summary  at  the  end  of  each  lesson. 


These  behaviors  differ  fro*  the  first  three  In  that  they  pertain  to  the  actual  presentation 
of  lesson  content,  although  preparation  for  the*  nay  begin  long  before  the  start  of  a  lesson. 
These  four  practices  also  represent  ways  of  establishing  a  clear  and  understandable  dialogue  with 
students.  To  "understand*  Is  to  receive  the  message  being  sent.  As  was  noted  with  the  three 

behaviors  above,  part  of  successfully  receiving  the  message  Is  being  ready  and  prepared  to 
receive  It.  Informing  the  learner  as  to  what  Is  expected,  providing  advanced  organization  and 
checking  for  and  reteaching  task-relevant  prior  learning  are  considered  ways  of  helping  to 
prepare  learners  to  receive  the  message.  In  addition  to  these  behaviors,  however,  researchers 
have  studied  ways  to  make  clear  the  message  Itself;  that  Is,  to  "package"  It  In  ways  that  make  It 
Intelligible  to  learners,  so  that  they  can  not  only  hear  and  Interpret  but  also  retain  what  Is 
being  said. 

A  frequently  reported  criticism  from  students  Is  that  they  "weren't  told"  or  "didn't 
understand"  what  to  do.  In  addition,  researchers  report  that  learners  frequently  are  afraid  to 
adeit  that  they  could  not  follow  the  directions  given,  either  because  they  could  not  remember  all 
that  was  said  or  It  was  said  so  quickly  and  matter-of-factly  that  the  most  Important  part  of  the 
directive  became  lost  among  less  relevant  detail.  The  point  of  this  fourth  behavior  Is 
directions  pertaining  to  completing  exercises,  reading  assignments,  homework,  and  drill  and 

practice  need  to  be  comunlcated  with  the  same  deliberateness  as  used  to  present  content. 
Typically,  It  has  been  found  that  Instructors  speed  up  and  speak  less  distinctly  when  giving 

directions  as  to  what  to  do,  how  to  proceed,  and  what  rules  to  follow  In  completing  work  than 

when  actually  teaching  the  content  to  which  the  directions  pertain.  When  learners  cannot  follow 
directions  as  to  how  the  Instructor  wants  them  to  become  engaged  In  the  learning  process  or 
complete  performance-oriented  work  assignments,  researchers  report  that  more  often  than  not  the 
students  silently  proceed  on  their  own,  many  times  missing  the  Intent  of  an  assignment.  The 
appropriate  recommendation  often  given  here  Is  to  slow  down  when  conveying  Instructions,  taking 
care  to  divide  directives  Into  steps  If  need  be,  and  checking  to  be  sure  each  step  Is  understood 
along  the  way. 

The  next  behavior,  knowing  the  ability  levels  of  the  students  and  teaching  at  their  levels, 
often  Is  reported  to  be  one  of  the  most  difficult  to  achieve.  Since  most  Instructors  are  likely 
to  have  students  of  several  levels  of  ability  (e.g.,  less  able,  average,  more  able)  represented 
In  their  classes,  they  are  likely  to  become  frustrated  by  an  Inability  to  teach  some  students. 
Nevertheless,  research  on  clarity  suggests  the  need  to  package  Instructional  stimuli  In  the  form 
of  oral  presentations,  visual  messages,  exercises  and  reading  assignments  that  are  at  the  current 
level  of  functioning  of  all  of  the  students.  To  accomplish  this.  Instructors  may  need  to  provide 
a  range  of  Instructional  stimuli  In  their  classrooms.  Demonstration  centers,  reference 
libraries,  different  typ«  of  pictorial  displays,  and  even  alternate  texts  and  exercises  that  tap 
into  abilities  slightly  above  and  below  the  average  learner  are  some  of  the  ways  researchers 
suggest  that  clarity  can  be  maintained  In  a  diverse  classroom. 

The  next  two  behaviors  for  achieving  clarity  both  occur  during  the  lesson,  and  both  are 
Intended  to  expand  upon  and  clarify  content  in  the  text  or  workbook.  The  point  of  these 
behaviors  Is  that  textbook  and  workbook  content  may  not  be  understood  completely  or  uniformly  by 
all  students  without  elaboration.  This  means  that  the  Instructor  must  enliven  the  content  by 
presenting  It  In  different  forms  that  can  highlight  its  most  Important  features.  The  research 
literature  suggests  that  oral  examples,  visual  Illustrations  and  practical  demonstrations,  if 
different  than  in  the  text,  measurably  increase  the  clarity  of  a  lesson  for  learners  who  are 
searching  for  different  ways  to  attach  meaning  to  the  lesson  and  relate  It  to  their  own 
experiences. 

The  research  literature  suggests  that  reviews  and  sunmarles  at  the  end  of  a  lesson  or 
Interspersed  throughout  the  lesson  can  have  much  the  same  effect.  The  research  also  suggests 
that  a  review  or  summary  should  not  be  a  simple  parroting  of  what  was  taught  In  some  abbreviated 
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form.  Instead,  an  effective  review  or  summary  presents  content  In  a  slightly  different  form  than 
that  In  which  It  was  Initially  presented,  thereby  elaborating  upon  and  organizing  It  differently 
for  efficient  storage  and  retrieval.  This  provides  learners  the  opportunity  to  “plug"  Into  the 
content,  not  only  at  a  different  time  but  In  a  different  manner  as  well.  Content  unlearned  or 
misunderstood  during  the  lesson  becomes  learned  and  clarified  during  sumarles  and  reviews  If  the 
sumury  and  review  go  beyond  a  simple  repetition  of  the  content  to  a  repackaging  of  It  In  a 
manner  efficient  for  retention  and  later  retrieval. 

Variety  In  the  Classroom 

Table  2  summarizes  some  of  the  behaviors  related  to  Instructional  variety. 

The  research  evidence.  The  primary  and  secondary  sources  providing  strong  or  promising 
research  evidence  for  the  subdimensions  of  variety  presented  In  Table  2  are  as  follows: 


Primary 

Armento,  B.  (1977).  Teacher  behaviors  related  to  student  achievement  on  a  social  science  concept 
test.  Journal  of  Teacher  Education,  28,  46-52. 

Bettencourt,  E.,  Glllett,  M.,  Gall,  M.,  6  Hull,  R.  (1983).  Effects  of  teacher  enthusiasm  training 
on  student  on-task  behavior  and  achlvement.  American  Educational  Research  Journal,  20, 
435-450. 

Bllgh,  D.A.  (1970).  The  case  for  a  variety  of  teaching  methods  In  each  lesson.  British  Journal 
of  Medical  Education,  202-209. 

Brophy,  J.,  4  Evertson,  C.  (1976).  Learning  from  teaching:  A  developmental  perspective. 

Boston:  Allyn  and  Bacon. 

Emmer,  E.  (1967).  The  effects  of  teacher  use  of  student  Ideas  on  student  verbal  Initiation. 
Unpublished  doctoral  dissertation.  University  of  Michigan. 

McConnell,  J.  (1977).  Relationships  between  selected  teacher  behaviors  and  attitudes/ 

achievements  of  algebra  classes.  Paper  presented  at  the  annual  meeting  of  the  American 
Educational  Research  Association,  New  York,  NY. 

Ryan,  F.  (1973).  Differentiated  effects  of  levels  of  questioning  on  student  achievement. 
Elementary  School  Journal,  41 ,  63-67. 

Ryan,  F.  (1974).  The  effects  on  social  studies  achievement  of  multiple  student  responding  to 
different  levels  of  questioning.  Journal  of  Experimental  Education,  42,  71  -75. 

Soar,  R.S.  (1966).  An  Integrative  approach  to  classroom  learning  (Report  for  NIMH  Projects  No. 
5-R11  MH01096  and  No.  7-R11  MH02045).  Philadelphia:  Temple  University.  (ERIC  Document 
Reproduction  Service  No.  ED033749). 

Tlkunoff,  W.,  Berliner,  D.,  6  Rlst,  R.  (1975).  An  ethnographic  study  of  the  forty  classrooms  of 
the  Beginning  Teacher  Evaluation  study  known  sample  (Tech.  Rep.  No.  75-10-5).  San  Francisco: 
Far  West  Laboratory. 

Wlnne,  P.  (1979).  Experiments  relating  teachers'  use  of  higher  cognitive  questions  to  student 
achievement.  Review  of  Educational  Research,  49,  13-50. 
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Table  2.  Effective  end  Ineffective  Indicators  for  Variety 
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Varies  types  of  questions  (e.g..  Always  asks  divergent,  opinion  Match  questions  to  the  behavior 

divergent,  convergent)  weekly  and  questions  (e.g,.  What  do  you  and  cooplexlty  of  the  lesson  ob- 

probes  (e.g.,  to  clarify,  to  think  about...?)  without  follow-  jectlve.  Vary  complexity  of 

solicit,  to  redirect)  dally.  up.  Or  overuses  convergent,  lesson  objectives  In  accord  with 

fact  questions.  the  unit  plan. 


Secondary 

Brophy,  J.,  1  Good,  T.L.  0986).  Teacher  behavior  and  student  achievement.  In  M.C.  Wlttrock 
(Ed.),  Handbook  on  research  on  teaching  (3rd  ed.,  pp.  328-374).  Mew  York:  Macmillan. 

Ounkln,  M.J.,  i  Biddle,  B.J.  (1974).  The  study  of  teaching.  New  York:  Holt. 

Good,  T.,  Biddle,  8.,  &  Brophy,  J.  (1975).  Teachers  make  a  difference.  New  York:  Holt. 

Gall,  M.D.  et  al.  (1978).  Effects  of  questioning  techniques  and  recitation  on  student  learning. 
American  Educational  Research  Journal,  1  5,  175-199. 

Hanley,  E.M.  (1970).  Review  of  research  Involving  applied  behavior  analysis  In  the  classroom. 
Review  of  Educational  Research,  40,  597-625. 

Lysakowskl,  R.S.,  i  Walberg,  H.J.  (1981).  Classroom  reinforcement  and  learning:  A  quantitative 
synthesis.  Journal  of  Educational  Research,  75,  69-77. 

O'Banlon,  O.R.,  S  Whaley,  D.C.  (1981).  Behavior  contracting:  Arranging  contingencies  of 
reinforcement.  New  York:  Springer  Publishing  Company. 

O'Leary,  K.,  t  Drabman,  R.  (1981).  Token  reinforcement  programs  In  the  classroom:  A  review. 
Psychological  Bulletin,  75,  379-398. 

Redfleld,  O.L.,  t  Rousseau,  E.W.  (1981).  A  meta-analysis  of  experimental  research  on  teacher 
questioning  behavior.  Review  of  Educational  Research,  51,  237-245. 

Rosenshlne,  B.  (1970).  Enthusiastic  teaching:  A  research  review.  School  Review,  78,  499-514. 

Rosenshlne,  B.  (1971).  Teaching  behaviors  and  student  achievement.  London:  National  Foundation 
for  Educational  Research  In  England  and  Wales. 

Rationale  for  the  behaviors  comprising  Instructional  variety.  The  following  section  provides 
an  overview  of  the  rationale  for  each  of  the  subcomponents  of  variety  based  upon  the  above-cited 
research  literature. 

As  noted  In  Table  2,  the  teaching  behavior  of  variety  Involves  six  subdimensions: 

1.  Using  attention-gaining  devices. 

2.  Showing  enthusiasm. 

3.  Varying  Instructional  activities  or  teaching  functions. 

4.  Mixing  rewards  and  reinforcers. 

5.  Varying  types  of  questions  and  probes. 

6.  Using  student  Ideas. 

The  first  behavior,  using  attention-gaining  devices.  Is  generally  considered  the  first 
Ingredient  of  a  good  lesson  plan.  Attention-gaining  devices  with  which  a  lesson  begins  can  take 
many  forms  Including  pictures,  audio/video  tapes,  demonstrations,  experiments;  they  may  also  have 
a  less  spectacular  quality  about  them,  such  as  the  posing  of  a  challenging  question,  presentation 
of  a  dlleflua  or  bewildering  situation,  or  even  silence  associated  with  a  unique  or  Interesting 
visual  display.  The  research  suggests  that  the  point  of  beginning  a  lesson  In  this  manner  Is  to 
stimulate  the  learner  In  a  way  different  than  he  or  she  has  become  accustomed  to  during  the 
previous  activity,  by  providing  a  change  In  the  type  of  Instructional  stimuli.  The  point  of  this 
Is  to  stimulate  not  only  the  receptive  modalities  of  sight  and  sound  but  also  the  cognitive 
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processes  associated  with  th«a.  without  this  awakening  and  conscious  change  fro*  the  mood  and 
tempo  of  an  earlier  activity  or  class,  the  learner's  attention  may  not  be  fully  directed  to  the 
Instructor  and  lesson.  Attention-gaining  devices,  therefore,  are  recommended  to  help  create 
natural  cycles  of  'highs*  and  'lows*  that  make  life  In  classroom  none  Interesting  and  less 
regimented  to  learners.  Although  It  Is  coamonly  recognized  that  points  of  high  curiosity. 
Interest  and  visual  Impact  can  seldom  be  sustained  for  long  periods,  the  research  suggests  that 
without  the*,  there  can  be  no  feeling  of  anticipation  which  Is  thought  to  be  a  precondition  to 
establishing  a  desirable  learning  set. 

One  of  the  most  comon  recommendations  from  the  research  literature  to  maintain  the  momentum 
of  rising  and  falling  cycles  of  Interest,  curiosity  and  excitement  In  a  classroom  Is  to  vary 
voice,  eye  contact,  and  gestures.  When  Interest  Is  waning  In  a  class  and  no  attention-getting 
device  Is  readily  available  to  pique  the  learners'  attention,  a  change  In  Intonation,  eye 
contact,  or  gesturing  Is  thought  to  be  In  order.  Although  Instructors  are  often  told  not  to  be 
classroom  actors,  they  are  also  told  that  much  can  be  learned  from  the  field  of  drama  that  Is 
applicable  to  the  classroom.  For  example,  plays,  like  lessons,  should  have  opening  and  closing 
acts  and  climaxes  and  anticlimaxes  In  between,  to  keep  the  audience's  attention.  How  well  an 
Instructor  plans  the  "script*— or  lesson— has  much  to  do  with  Its  attention-getting  quality. 
Aside  from  the  natural  variety  In  topics  and  Instructional  activities  a  lesson  may  contain, 
variety  can  be  enhanced  by  accentuating  the  lesson's  high  points  and  bolstering  Its  low  points 
with  changes  In  voice,  eye  movement,  and  body  movement.  Simply  put,  the  research  literature 
suggests  that  voice  Inflection,  eye  contact,  and  positioning  In  the  classroom  should  change 
often,  especially  during  the  high  point  (e.g.,  question  and  answer)  and  low  point  (e.g.,  rote 
recitation).  Raising  the  voice  slightly,  shifting  one's  view  from  the  front  of  the  class  to  the 
back,  or  moving  to  a  new  location  In  the  classroom  during  these  times  Is  often  considered  an 
Important  dimension  of  Instructional  variety. 

There  Is  little  question  among  classroom  researchers  that  effective  teaching  Involves  many 
different  classroom  activities.  Research  Indicates  that  an  Instructor  who  does  nothing  but 
lecture  for  an  entire  period,  or  who  only  engages  learners  In  prolonged  seatwork,  or  who  does 
nothing  but  expose  learners  to  attention-gaining  devices  would  have  difficulty  achieving 
successful  unit  outcomes,  especially  at  higher  levels  of  behavioral  complexity  (e.g., 
application,  analysis,  synthesis,  and  decision  making).  Although  researchers  would  generally 
agree  that  some  classes  from  time  to  time  should  emphasize  a  single  activity,  the  majority  of 
classes  should  offer  some  variety  In  the  activities  with  which  an  Instructor  presents 
Instructional  stimuli.  This  means  that  some  lecturing  should  be  mixed  with  some  questions  and 
answers  (which  may  Include  work  at  the  board),  which  may  be  followed  by  guided  practice  In  which 
the  learners  are  asked  to  make  their  first  attempt  at  responding  appropriately. 

These  types  of  Instructional  routines,  when  thought  out  In  advance  and  well  executed,  provide 
the  variety  needed  for  effective  teaching.  In  this  sense,  the  purpose  of  the  variation  In 
Instructional  activities  Is  not  so  much  to  gain  the  attention  of  the  learner  but  to  allow  the 
learner  to  cognitively  process  the  material  In  a  variety  of  different  ways— for  example,  by 
listening  (lectures),  by  seeing  (textbook),  and  by  doing  (workbook).  Changing  the  modality 
(hearing,  seeing,  doing)  provides  the  learner  with  the  redundancy  often  needed  for  mastery  and 
presents  the  stimulus  material  In  different  contexts,  emphasizing  different  cognitive  abilities, 
which  different  learners  possess  to  varying  degrees.  Therefore,  a  lesson  plan  that  Includes  some 
combination  of  lecture,  discussion,  question  and  answer,  guided  practice,  and  Independent 
seatwork  generally  Is  preferred  to  one  that  exclusively  emphasizes  only  one  of  these 
Instructional  alternatives.  Varying  an  Instructional  routine  across  two  or  more  of  these  formats 
and  varying  the  combinations  chosen  across  days  are  generally  believed  to  add  the  Important 
dimension  of  variety  to  a  unit  plan. 

One  of  the  first  things  often  noticed  by  researchers  observing  In  classrooms  Is  how  the 
Instructor  Interacts  with  students.  Most  obvious  among  the  types  of  Interchanges  that  occur  and 
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have  been  reported  In  the  research  literature  are  those  that  bestow  reward  or  punishment. 
Perhaps  because  rewards  are  so  laportant  In  dally  life,  observers  tend  to  be  acutely  aware  of  the 
quantity  and  quality  of  the  rewards  given  to  learners.  Verbal  praise  Is  the  aost  frequently  used 
type  of  reward  In  the  classroom.  The  research  suggests  that  the  reinforcing  effect  of  verbal 
praise,  however,  depends  on  the  variety  with  which  It  Is  administered  and  the  other  types  of 
rewards  with  which  It  Is  associated.  Most  researchers  agree  that  the  phrase  "That's  good"  or 
"That's  correct*  when  repeated  In  the  exact  saae  voice  hundreds  of  times  across  problems  and 
learners  Is  not  reinforcing;  that  Is,  it  Is  not  perceived  by  the  learners  as  a  reward.  The 
phrase  soon  becomes  meaningless  to  the  learner  and,  thereby,  loses  its  ability  to  reinforce  the 
behavior  Intended.  The  research  literature  points  out,  therefore,  the  importance  of  varying  not 
only  the  type,  amount  and  Intensity  of  the  verbal  praise  given  but  of  branching  out  and 
experimenting  with  other  types  of  rewards.  What  Is  rewarding  to  one  student  may  not  be  rewarding 
to  another.  Using  a  learner's  correct  answer  as  a  model  for  the  next  problem,  having  the  learner 
retrace  for  the  class  how  the  correct  answer  was  obtained,  or  having  peers  comment  on  the 
correctness  of  a  learner's  answer  are  some  of  the  suggested  ways  that  verbal  praise  can  be  varied 
to  provide  rewards  that  are  more  meaningful  to  the  student. 

Another  dimension  of  variety  which  appears  In  the  research  literature  pertains  to  questioning 
and  probing.  The  art  (or  science)  of  questioning  and  probing  Is  thought  to  be  one  of  the  most 
Important  skills  of  an  effective  Instructor.  It  Is  commonly  stated  In  the  literature  that  the 
variety  often  conveyed  to  learners  will  to  a  large  degree  be  determined  by  a  flexible  use  of 
questions  and  probes.  The  purpose  of  these  two  related  techniques  is  to  elicit  from  the  student 
a  response  (sometimes  any  response)  that  then  can  be  shaped  and  formed  Into  a  better  or  more 
complete  response.  Such  questions,  however,  are  rarely  considered  ends  In  themselves  but. 
Instead,  are  often  a  precursor  to  subsequent  learning.  That  Is,  they  provide  a  vehicle  for 
engaging  the  student  In  the  learning  process  by  getting  the  learner  to  act  on,  work  through,  or 
think  about  the  material  that  has  been  previously  presented. 

Although  questions  are  tools  for  engaging  students  In  the  learning  process,  the  research 
suggests  that  to  be  effective,  they  must  be  administered  flexibly  and,  often,  followed  up  with 
other  questions  that  attempt  to  probe  Into  the  glib,  superficial  or  Inadequate  responses  that  are 
sometimes  Initially  given  to  a  question.  Probes,  therefore,  are  questions  that  follow  questions, 
carefully  crafted  to  deepen,  enrich,  and  extend  an  earlier  response. 

Probes  are  used  to  elicit  a  clarifying  response  to  an  earlier  question,  solicit  new 
Information  related  to  an  earlier  question,  or  redirect  the  learner  Into  a  more  productive  area. 
Each  of  these  uses  of  probes  Is  thought  to  offer  a  source  of  variety  that  can  add  momentum  to 
questioning  behavior,  as  when  a  solicitation  for  new  Information  Is  followed  by  a  request  for 
clarification  which.  In  turn.  Is  followed  by  a  redirection  to  a  new,  more  productive  avenue  of 
thinking.  It  Is  recommended  these  cycles  of  probes  be  executed  In  "waves"  which  return  again  and 
again  to  force  learners  to  act  upon  and  reshape  their  responses— and  most  Importantly,  their 
thinking. 

The  research  also  suggests  that  there  should  be  variety  as  well  In  the  type*  of  questions 
asked,  alternating  between  convergent  and  divergent,  although  not  necessarily  to  equal  degrees. 
Convergent  questions,  which  usually  have  a  single  right  answer,  are  most  cowonly  associated  with 
the  goals  of  direct  Instruction  used  In  the  teaching  of  facts,  rules,  and  sequences.  Divergent 
questions,  which  may  have  many  right  answers,  are  most  comnonly  associated  with  the  goals  of 
Indirect  Instruction  used  In  the  teaching  of  problem  solving.  Inquiry,  and  discovery  learning. 
The  Importance  of  these  two  types  of  questions,  however.  Is  not  so  much  their  association  with 
direct  or  Indirect  types  of  Instruction  as  It  Is  their  capacity  for  eliciting  behavior  at  lower 
and  higher  levels  of  behavioral  complexity.  Convergent  questioning  often  Is  recomnended  for 
eliciting  behavior  at  the  knowledge,  comprehension,  and  application  levels  and  divergent 
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questioning  often  Is  best  suited  for  eliciting  behavior  at  the  analysis,  synthesis,  and 
evaluation  levels  (Bloom,  Englehart,  Hill,  4  Kranthwohl,  1956). 

A  final  dimension  of  variety  Involves  the  use  of  student  Ideas.  This  source  of  variety  Is 
closely  related  to  divergent  types  of  questions,  which  have  more  than  a  single  correct  answer. 
Researchers  have  found  that  there  may  be  no  greater  variety  than  that  which  occurs  In  learner 
responses  to  divergent  questions.  Every  learner  Is  an  Individual  In  his  or  her  own  right,  which 
can  aake  for  considerable  diversity  In  the  responses  obtained  for  divergent  questions.  It  Is  the 
proper  use  of  such  diversity  that  research  has  suggested  Is  linked  to  effective  teaching. 
Diverse  learner  responses  can  be  a  problem  as  well  as  a  benefit.  In  that  unexpected  or 
dlfflcult-to-evaluate  responses  can  soawtloes  put  the  Instructor  on  the  defensive.  A  not  too 
uncomon  response  among  Inexperienced  Instructors  Is  to  Invent  a  correct  answer  to  end  the 
awkwardness  of  not  knowing  how  to  respond  to  an  unusual  and  unexpected  response.  According  to 
research,  this  would  not  be  a  proper  use  of  divergent  questioning,  since  one  of  the  purposes  of 
divergent  or  "open"  questions  Is  to  Incorporate  student  Ideas  and  participation  Into  the  lesson. 
The  key  to  eliciting  learner  Ideas  and  opinions  Is  to  make  their  Ideas  and  opinions  useful  to  the 
goals  of  the  lesson.  The  research  suggests  that  soliciting  Individual  learner  Ideas  and  opinions 
seldom  is  considered  sufficient,  if  It  cannot  be  used  to  create  greater  understanding  for  all 
learners  In  the  class. 

Using  this  source  of  variety  effectively  Mans  taking  the  contributions  of  Individual 
learners  and  building  nore  general  concepts,  patterns,  and  abstractions  from  the*  that  are 
relevant  to  the  goals  of  the  lesson. 


Task  Orientation  In  the  Classroo* 


Table  3  summarizes  $o*e  of  the  behaviors  related  to  task  orientation. 

The  research  evidence.  The  primary  and  secondary  sources  providing  strong  or  proalslng 
research  evidence  for  the  subdlMnslons  of  task  orientation  shown  In  Table  3  are  as  follows: 


Primary 

Arehart,  J.  (1979).  Student  opportunity  to  learn  related  to  student  achievement  of  objectives  in 
a  probability  unit.  Journal  of  Educational  Research,  72,  253-269. 

Borg,  W.  (1979).  Teacher  coverage  of  academic  content  and  pupil  achleveMnt.  Journal  of 
Educational  Psychology,  71,  635-645. 

Brophy,  T.,  4  Evertson,  C.  (1976).  Learning  fro*  teaching:  A  deve1op*ental  perspective. 

Boston:  Allyn  and  Bacon. 

Cooley,  W.,  4  Lelnhardt,  G.  (1980).  The  Instructional  dlMnslons  study.  Educational  Evaluation 
and  Policy  Analysis,  2,  7-25. 

Dunkln,  M. ,  4  Doehav,  S.  (1980).  A  replication  study  of  unique  and  joint  contributions  to 
variance  In  student  achievement.  Journal  of  Educational  Psychology,  72  ,  394-403. 

Emmer,  £.,  Evertson,  C.,  Sanford,  J.,  Clements,  8.,  4  Worsham,  M.  (1984).  Classroom  management 
for  secondary  teachers.  Englewood  Cliffs,  NJ:  Prentice-Hall. 
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Being  task-«riented  Poor  task  orientation 

(An  effective  instructor) _ (An  ineffective  instructor) _ (Reco— ended  behavior! 

Develops  unit  and  lesson  plans  Develops  lessons  lastly  froa  personal  Key  each  lesson  to  a  unit  plan, 

that  emphasize  the  nost  critical  or  student  Interests.  Breadth  and  depth  the  curriculum  guide  and  the 

features  of  the  curriculum  or  of  lesson  content  fail  to  distinguish  text  to  test  its  relevance. 


I. 

S 

o 

2 


w 

& 


2  ^ 

&. 
o 


I  I—  (. 

—  r  'Sc  m  s 

«  L  O  I  k.  <4. 

U  3  V  ^  C  4)  fl 

—  o  c  •*>  o  «*» 

44  3  O  C  3  k. 

—  P  2  3  O  O 

k.  C  5^0 

0  3  3  44  44  II 

VI  •  £ 

SC  -3  VI  O 

■v  41  Jd  •*- 

B  44  V)  3 

■e-  <4-  O  3  C 

X  O  >  44 

SOI  O 

k.  *3  —  44 

3  3 

01  O  01  C  VI  • 

-C.QO.HC 

c  —  vi  o 

VI  U  C  44  |H  VI 

:=  2L  3  y  * 


vi  X 

2  5 

2  2 


2“S 
£ 

O  3 

O 


5  3  2 


O  44 

U  X 
3 

£>  w 
3  P 
3  C 
C  3 

o 

S  4! 


•  1 

*3 

If 


il 


« 

k. 

c  -£  1 
'  £  = 
£  «  a 

c  c  k. 

3  k. 

e  3 

-C  3  O 

u  u 

3  3 

UJ  3  C 

>  4-* 

•  <-*  o 

£  2  ** 
3  ^  JO 
4-A  C  U 
O  3 

3  C 

3  e 
0  3 
-  oi 


3  3 

3  — 


lui 

ti  5  s 

C  4J  C 
'T'  O  V 


c 

3  O 
C  3  — 
3  C  d-l 
V  o 
3  3 

i  ?£ 

41  VI 

5  I  “ 

vi  O 


—  3  3 

«  S  5 

3  3  0 

£•5  i 

>  -X  >t 
3  VI  v— 

O  2*3 

*•  -  B 
I  5  § 

3  k.  3 
•M  3  8 

4-» 

«c  o  w 


I 

O  ‘  “ 

3 

c 

3 


I! 


3 

C 


«  y  e 

—  C  VI  — 


?  J2  £ 


—  ♦*30 


3  V 
& 


44  I  k 

3  U  VI 


VI 

3 


2  £ 

C  k.  —  3  O 

e  a  ‘ 

_  O 

2  ~  ° 

U 

2 


*8  P  — 


-s? 

£21 


5  £ 


9  a  a 


3  — 
—  3 


—  o  *» 

&  I  3 
a  3  <w 
3  V-  3 
vi  a.  3 


• 

VI 

v  V  3 
3  VI  C 

w  2  10 

O  VI 

VI  c  O 

s  ^  > 

f  k  3 
+-»  O  C 
C  <*»  3 
3  C 


—  U  44 

3  3  I 

£,  t£o  I 

■V-  3  C 

—  C  C  +4 

lilt: 

VI  •—  O  O 


S3  O 
3  4-1 


3  > 
£  3 


3  4J 
O  X 
3 


3  3 
—  3 


44  O 
■v-  3 


U 

3  O 


O 

U  I 
C  3  O 
—  »*-  3 


4-»  C 

C  M  O 

Ol 

3  3  — 

3  A  4-» 

_  O 

O  «4-  o 


k. 

o 


8 


V  VI  W 

C  3  — 
3  4-» 

44  3 

C  3 

O  — 


& 

>1 


^  8 


3  3 

u  e 

*  3 


CUC 
O  J  4j 

♦4  —  C 

U  3  O 

2  =  ° 

4-»  — 

VI  3  — 

c  c 

•*■3  3 

Q. 

4->  4->  >, 

*  s  T 

C  4-»  k. 

—  CO 

3  0  4- 

u 

3  C 
VI  —  O 

m  *  2 

3  4-> 

>  k.  VI 

—  O  C 


•  • 

c  o  •— 

U  C  ^  3 

C  V.  *>  C 

2  3  3  e 

C  5  -  k.  2  tx 

—  —  k.  o  3  u 

>  3  C  —  3 

3  —  3  —  >  C 

—  4J  —  B  3  3 

3  U  U  C  3 

3  3  *3 

3  C  *3  4- 

C  ■  4-»  CD  C 

U  O  —  •  3  k. 

VI  O  2  3  O 


VI 


VI 


c 

C  3  3  +J  V!  43 

vi  »—  C  3 

—  U  3  3  4J  >, 

—  C  >  C 

C  k.  3  3  k.  3 

3  0  O  B 

4-*  c  3  — >  5 

VI  3  —  —  3  3 

UJ  B  Ol  C  S  VI 


c 

>i 

X> 

VI 

o 

*o 

3 

C 

3 

1 

e 

44 

u 

u 

T3 

3 

O 

— 

2 

a. 

3 

Ol 

4- 

1 

s  § 

k. 

VI 

2 

k. 

>> 

3 

o 

3 

-a 

o 

c 

o 

Ll. 

3 

p 

C 

44 

r— 

o 

44  •*. 

o 

44 

T3 

^3 

3 

u 

44 

3 

44  44 

• 

*»— 

C 

C 

•o 

C 

r-» 

C 

c 

3  U 

c 

44 

VI 

o 

O 

3 

4- 

C 

3 

o 

o 

3 

o 

U 

e 

3 

X 

C 

o 

C 

VI 

>1 

3 

3 

c 

VI 

u 

3 

U 

o 

44 

VI 

—  *o 

VI 

k. 

u 

o 

44 

c 

VI 

P 

3 

VI 

— 

U 

3 

44  C 

VI 

44 

44 

VI 

44 

a. 

£ 

44 

2 

C  3 

3 

VI 

44 

VI 

3 

k. 

c 

>i 

VI 

u 

3 

U 

c 

U 

3 

44 

3 

o 

3 

2 

44 

Ol 

3  VI 

VI 

3 

3 

o 

U 

o 

VI 

c 

a*  3 

*»— 

k. 

m 

<*- 

VI 

3 

44 

44 

C 

3  — 

■P 

*o 

a. 

a 

e 

VI 

3 

VI 

> 

t  2 

C 

Ol 

£* 

3 

C 

-J 

c 

3 

c 

3 

■o 

u 

• 

U 

44 

44 

Ol 

c 

3 

3 

U 

? 

3 

c 

c 

«  • 

3 

VI 

3 

44 

> 

3 

• 

k. 

3 

u 

3 

e  VI 
Ol  44 

2 

2 

— 

2 

-X 

3 

3 

c 

P 

"SI 

♦4  »  3 
3  3  3 

C 

VI  3  3 
3  C  VI 

c  c 

3  .X  3 
V  lA  4- 
44  3 

<  4J  O 


k.  3  3 

VI  c 
3  U 
3  >  3 

0—3 

C  4-»  4-» 

4-»  U 

S3  O 
-M 


5S: 

4-*  k. 
VI  3  3 
3  a. 

8  •  ai 

3  VI  3 
3  +>  O 
cr  a.  u 

3  3  C 

U)  U  V 


Si  3 

—  Ql  2 


3  k. 

- 

Soils 

V  L  Jf 

SOI 
C  &-  C  C 

U  O  u  — 

VI  *— ■  — >  f—  > 

03  C  43  C  Q 

X  2  S  2  S 


I 

3 

C 


g  c  c  —  t  iv- 

4-  3  a.  u  —  4-»  c* 

o  i  —  a.  — 


* 

s 

3 
9  C 

fi 

&  U 


I 

-a 

o 


5  8 


k. 

& 

2 


c  u 

3 

^  N  ^  - 


8 


3 

> 

2  . 

a. 

k. 

o 


2  I 

3 

•  U 

•  3 
Ol 

•  3 

3  oi 


2 

2 


3 

2 


I  s 

S  3 

?  S 


o  ^ 


—  >>  C  3 

4J  VI  o  C 


VI  C  U 
4J  3  — 

U  — 

3  VI  3  3 

H-  3  C 

U  3 
PI  C  O 
C  3  ^  VI 

—  3  C 

4-»  O*  C  U 

—  303 

U  VI  —  ^ 

—  4J  4-» 

—  C  U  3 

3  0  3  0. 

k  V  44  •  • 

O  U  VI  VI  VI 

^  3  e  v  e 

—  o  o 


Ol 
c  • 

44  3 

2  — 

vi 

VI  44 

§  I 

U  3 

4J 

3  5 

C  -  _ 

4-»  3  C  — 


»  VI 

3  C- 
—  O 
>  — 
3  vi 


>i  vi 


k. 

O 


o  —  — 


o 

44 

l/l 


u 

u  2  — 

3  4-4  44 


O  C  44  y  44 

—  3  U  c  u 

SO  3 
U  k. 


—  4-*  *—  VI  U  —  k. 

3  C  3  C  3  U  44 


3  k. 


U  VI 

2  3  — 

r—  3  Ol  VI 

44  3  C  C  C 

VI  4-  —  —  3 


3  ^ 

o  8 

44 

VI  ^ 
T3  k. 

—  3 

—  3 
3  — 
CO  U 


2  £ 
3 


3  u 
3  3 

2  Q 


18 


Fltz-GIbbon,  C.,  i  Clark,  K.  (1982).  Tine  variables  In  classroom  research:  A  study  of  eight 

urban  secondary  school  mathematics  classes.  British  Journal  of  Educational  Psychology,  52, 
301-316. 

Smith,  L.  (1979).  Task-oriented  lessons  and  student  achievement.  Journal  of  Educational 
Research,  73,  16-19. 

Stallings,  J.,  A  Kaskowltz,  0.  (1974).  Follow  through  classroom  observation:  1972-1973  (SRI 
Project  URU-7370).  Stanford,  CA:  Stanford  Research  Institute. 


Secondary 


Block,  J.H.  (Ed.).  (1971).  Mastery  learning.  New  York:  Holt,  Rinehart,  and  Winston. 

Brophy.  J.,  4  Good,  T.  (1974).  Teacher-student  relationships:  Causes  and  consequences.  New 
York:  Holt,  Rinehart,  and  Winston. 

Clark,  C.M.,  6  Peterson,  P.L.  (1986).  Teachers'  thought  processes.  In  Merlin  C.  wittrock  (Ed.), 
Handbook  of  research  on  teaching  (3rd  ed.,  pp.  225-296).  New  York:  Macalllar. 

Duncan,  M.J.,  A  Riddle,  8.J.  (1974).  The  study  of  teaching.  New  York:  Holt,  Rinehart,  and 
Winston. 

Karwelt,  N.  (1983).  Time  on  task:  A  research  review  (Report  No.  332).  Baltimore,  J®:  Johns 
Hopkins  University,  Center  for  Social  Organization  School. 

Rosenshlne,  B.  (1971).  Teaching  behaviors  and  student  achievement.  London:  National  Foundation 
for  Research  In  England  and  Wales. 

Rosenshlne,  B.  (1983).  Teaching  functions  In  Instructional  programs.  The  Elementary  Schobl 
Journal .  4,  335-351. 

Rationale  for  the  behaviors  comprising  task  orientation.  The  following  Is  an  overview  of  the 
rationale  for  each  of  the  subcomponents  of  task  orientation. 

As  noted  In  Table  3,  the  teaching  behavior  of  task  orientation  Involves  five  subdimensions: 

1.  Organizing  lessons  In  a  manner  that  emphasizes  the  most  critical  features  of  the 
curricula  guide  and  text. 

2.  Handling  administrative  and  clerical  Interruptions  efficiently. 

3.  Stopping  or  preventing  Inattention  or  off-task  behavior,  with  a  minimum  of  class 
disruption. 

4.  Selecting  the  most  appropriate  Instructional  model  for  the  objectives  being  taught. 

5.  Establishing  systematic  cycles  of  review,  feedback,  and  testing. 

Research  suggests  that  one  of  the  most  effective  ways  of  assuring  a  task  orientation  is  to 
base  lessons  on  the  curriculum  guide  and  adopted  text.  Although  all  lesson  content  Is 
predetermined  In  the  military  classroom,  this  is  not  to  say  the  effective  implementation  of  this 
content  in  the  classroom  is  guaranteed.  Researchers  have  found  the  temptation  is  Indeed  great 
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among  classroom  Instructors  to  "reinterpret"  adopted  curriculum  and  textbook  content  Into  topics 
of  personal  Interest,  topics  about  which  an  Instructor  may  be  most  knowledgeable,  or  topics  which 
learners  appear  most  eager  to  "hear  about."  Although  personalizing  the  curriculum  so  that  It  can 
be  related  to  the  subjective  experience  of  learners  receives  support  In  the  literature,  this 
reformulation  must  emphasize  the  most  critical  features  of  the  curriculum  and  text.  Generally, 
accenting  adopted  curriculum  topics  with  personal  experiences,  examples,  and  illustrations  from 
outside  the  text  or  even  outside  the  curriculum  are  considered  earmarks  of  effective  teaching  If 
these  experiences,  examples,  and  Illustrations  are  related  In  meaningful  ways  to  the  established 
curriculum.  Developing  unit  plans  In  which  personal  experiences,  examples,  and  Illustrations 
from  outside  the  text  or  curriculum  are  checked  for  their  relevance  to  emphasizing  the  most 
critical  features  of  the  curriculum  Is  generally  considered  an  important  step  In  maintaining  an 
effective  task  orientation. 

Another  behavior  considered  to  be  an  Important  part  of  an  Instructor’s  task  orientation 
pertains  to  how  administrative  and  clerical  Interruptions  are  handled.  In  extensive  studies  of 
more  effective  and  less  effective  classroom  teachers,  researchers  found  this  behavior  to  be  among 
the  most  Important  In  determining  an  Instructor's  task  orientation  for  the  entire  class  period. 
Researchers  learned  from  these  studies  that  once  an  Instructor  turned  from  an  Instructional  task 
(e.g.,  lecture)  to  a  nonlnstructlonal  one  (e.g.,  passing  out  materials,  checking  for  supplies). 
It  was  Increasingly  difficult  to  return  to  the  Initial  state  without  devoting  still  more  time  to 
getting  learners  back  on  task.  Thus,  not  only  was  time  spent  on  the  nonlnstructlonal  activity 
but  also  sometimes  as  much  time  was  spent  getting  learners  to  return  to  their  original  state  of 
attention.  This  has  led  to  the  recommendation  to  Instructors  to  pre-organlze  as  many  clerical 
tasks  as  possible  before  class  and  establish  procedures  for  dealing  with  unexpected 
Interruptions.  Taking  time  to  collate  pages  of  a  handout,  staple  them  and  pass  them  out  In  the 
middle  of  an  Instructional  activity,  for  example,  often  was  seen  as  wasteful  of  valuable 
Instructional  time,  especially  when  these  activities  could  have  been  completed  before  the 
Instruction  began.  Also,  placing  supplies  and  handouts  on  the  desks  of  learners  before  class  is 
often  a  recommended  strategy  to  Increase  an  Instructor's  task-oriented  behavior. 

A  third  behavior  for  maintaining  an  effective  task  orientation  Inyolves  stopping  or 
preventing  Inattention  or  off-task  behavior,  with  a  minimum  disruption  to  the  class.  Generally, 
the  research  suggests  that  the  significance  of  off-task  behavior  should  be  deemphaslzed  at  the 
time  It  occurs  by  the  way  the  Instructor  deals  with  It.  This  does  not  mean  that  the  instructor 
Ignores  or  deals  half-heartedly  with  Inattention  or  off-task  behavior,  but  that  the  instructor 
minimizes  its  consequences  to  the  class  at  the  time  it  occurs  by  deferring  a  discussion  of  the 
consequences  to  the  student  exhibiting  the  off- task  behavior  to  a  later,  nonlnstructlonal  time. 
In  other  words,  the  research  suggests  that  the  effective  Instructor  deals  with  the  offense 
inmedlately  but  deals  with  the  consequences  of  the  offense  at  a  time  and  In  a  context  chosen  by 
the  Instructor.  Also,  the  effective  Instructor  establishes  and  comnunlcates  rules  before  the 
start  of  a  course.  Indicating  the  precise  procedures,  disincentives  and  punishments  to  be  Invoked 
when  certain  offenses  occur.  An  effective  task  orientation,  therefore,  suggests  that  an 
efficient  system  for  handling  the  punishment  phase  of  any  off-task  or  Inattentive  behavior  be 
established  and  cammjnlcated  at  the  beginning  of  a  course  of  instruction. 

In  addition  to  Increasing  the  amount  of  class  time  that  Is  devoted  to  Instruction  by 
pre-organizing  administrative  and  clerical  chores  and  establishing  procedures  for  dealing  with 
off- task  behavior,  an  effective  task  orientation  Includes  the  efficient  use  of  the  time  that  is 
devoted  to  Instruction.  In  this  sense,  the  use  of  Inefficient  Instructional  strategies  are  often 
considered  as  wasteful  of  instructional  time  as  are  adnlnlstratlve  chores  and  off-task  behavior. 
Inefficient  instructional  strategies  are  thought  to  be  especially  damaging  to  the  attainment  of 
instructional  goals.  Unless  considerable  thought  Is  explicitly  given  to  the  match  between  an 
instructor's  objectives  and  the  Instructional  activities  being  used  to  promote  those  objectives, 
the  Instruction  may  look  but  not  actually  be  task-oriented. 
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Research  on  the  match  between  an  Instructor's  objectives  and  the  teaching  activities  most 
appropriate  for  promoting  those  objectives  has  led  to  the  development  of  the  direct  and  Indirect 
models  of  Instruction.  The  functions  of  each  are  summarized  In  Tables  4  and  5,  respectively. 
These  models,  generally,  are  considered  to  represent  the  means  to  efficiently  teach  two 
distinctly  different  types  of  behaviors.  These  two  types  of  behaviors,  representing  the 
acquisition  of  facts,  rules  and  action  sequences  and  the  learning  of  concepts,  abstractions  and 
patterns,  represent  the  range  of  behavioral  complexity  found  In  almost  every  subject  and 
Instructional  level.  Researchers  report,  however,  that  It  Is  not  uncommon  to  observe  Instructors 
attempting  to  teach  concepts  In  the  context  of  drill  and  practice  (direct  Instruction)  and  to 
teach  facts  In  the  context  of  an  Inquiry-type  discussion  (Indirect  Instruction).  Although  both 
outcomes  could  be  achieved  In  either  context,  the  teaching  of  concepts  by  rehearsing  facts  (e.g., 
memorizing  parts  of  a  jet  engine  or  reading  the  names  on  a  parts  list)  would  not  be  considered  an 
efficient  means  of  learning  the  concept  of  "thrust*  or  "Inventory."  The  teaching  of  facts  seldom 
allows  for  the  generalizations  and  discriminations  required  for  learning  concepts;  likewise,  the 
teaching  of  concepts  falls  to  emphasize  the  rules  and  sequences  required  for  the  acquisition  of 
facts.  Therefore,  the  Importance  of  matching  the  type  of  learning  outcome  to  be  achieved  with 
the  Instructional  model  which  most  efficiently  accomplishes  the  outcome  Is  considered  another 
Important  behavior  for  establishing  an  effective  task  orientation. 

A  final  behavior  for  task  orientation  involves  establishing  cycles  of  weekly  and  monthly 
review  and  testing.  These  cycles  are  built  around  clearly  definable  goals  (e.g.,  a  test  at  the 
end  of  the  month,  a  review  session  the  next  week,  a  laboratory  evaluation  at  the  end  of  the 
unit).  These  are  the  types  of  "products"  that  research  suggests  should  be  made  visible  to 
learners  and  toward  which  Instructional  activities  should  gradually  build.  It  has  been  suggested 
that  these  end  products  can  create  natural  cycles  of  rising  and  falling  Intensity,  enthusiasm  and 
expectation,  with  the  high  point  of  the  cycle  being  just  before  the  expected  event  Is  to  occur 
and  the  low  point  Immediately  afterward,  marking  the  beginning  of  a  new  cycle.  Several  different 
cycles  may  occur  concurrently,  as  when  some  Instructional  activities  are  preparing  learners  to 
complete  a  laboratory  assignment  or  practice  exercise  next  week,  while  other  Instructional 
activities  are  preparing  learners  to  do  well  on  a  written  test  to  be  given  at  the  end  of  the 
month.  Thus,  different  cycles  can  be  put  in  place  for  tests,  assignments,  .and  major  projects, 
staggered  in  such  a  way  as  to  have  one  cycle  near  Its  highest  point  when  another  Is  near  Its 
lowest.  The  research  suggests  that  this  can  create  a  high  level  of  Intensity,  enthusiasm,  and 
expectation  pertaining  to  some  clearly  definable  goal  at  all  times,  If  these  cycles  are  made 
visible  to  students  as  an  organizational  framework  for  study  and  review. 


Engagement  In  the  Learning  Process 

Table  6  suumarlzes  some  of  the  behaviors  related  to  a  learner's  engagement  in  the  learning 
process. 

The  research  evidence.  The  primary  and  secondary  sources  providing  strong  or  promising 
research  evidence  for  the  subdimensions  of  student  engagement  behavior  shown  In  Table  4  are  as 
follows: 

Primary 

Battle,  E.S.  (1965).  Motivational  determinants  of  academic,  task  persistence.  Journal  of 
Personality  and  Social  Psychology,  2,  209-218. 

Brophy,  J.,  i  Evertson,  C.  (1976).  Learning  from  teaching.  Boston:  Allyn  and  Bacon. 

Crawford,  J.  (1983).  A  study  of  instructional  processes  in  Title  I  classes:  1981-82.  Journal 
of  Research  and  Evaluation  of  the  Oklahoma  City  Public  Schools,  13  ( 1 ) . 
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Table  4.  Some  Direct  Instruction  Functions3 


1.  Reviews  dally,  checks  previous  day's  work,  and  reteaches  If  necessary. 

Checks  homework. 

Reteaches  areas  where  there  were  student  errors. 

2.  Structures  and  presents  new  content. 

Provides  overview. 

Proceeds  In  small  steps  (If  necessary),  but  at  a  rapid  pace. 

If  necessary,  gives  detailed  or  redundant  Instructions  and  explanations. 
Phases  In  new  skills  while  old  skills  are  being  measured. 

3.  Guides  student  practice. 

Uses  a  high  frequency  of  questions  and  overt  student  practice 
(from  teacher  and  materials). 

Provides  prompts  during  Initial  learning  (when  appropriate). 

Allows  all  students  a  chance  to  respond  and  receive  feedback. 

Checks  for  understanding  by  evaluating  student  responses. 

Continues  practice  until  students  respond  quickly  and  accurately. 

Achieves  success  rate  of  80S  or  higher  during  Initial  learning. 

4.  Provides  feedback  and  correctives  (and  recycling  of  Instruction,  If  necessary). 

Provides  feedback  to  students  (particularly  when  they  are  correct 
but  hesitant). 

Receives  feedback  from  students  as  to  whether  corrections  and/or 
reteaching  Is  necessary. 

Corrects  by  simplifying  question,  giving  clues,  explaining  or 
reviewing  steps,  or  reteaching  last  steps. 

When  necessary,  reteaches  using  smaller  steps. 

• 

5.  Uses  Independent  practice  to  elicit  student  responses  that  are  correct 

and  automatic. 

Assigns  seatwork. 

Employs  utilization  to  achieve  automatlclty  (practice  to  overlearning). 
Monitors  students  to  ensure  engagement  during  seatwork. 

Achieves  success  rate  of  95S  correct  or  higher. 

6.  Provides  weekly  and  monthly  reviews. 

Reteaches,  If  necessary. 

aAdapted  from  R.  Rosenshlne  (1983). 
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Table  5.  Sow  Indirect  Instruction  Functions 


1.  Provides  a  means  of  organizing  content  In  advance. 

Provides  advanced  organizers  and  conceptual  frameworks,  which  serve 
as  'pegs*  on  which  to  hang  key  points  that  guide  and  channel  thinking 
to  the  most  productive  areas. 

Allows  for  concept  expansion  to  higher  levels  of  abstraction. 

2.  Provides  conceptual  movement  using  Inductive  and  deductive  methods. 

Focuses  generalization  to  higher  levels  of  abstraction  by: 

Inductive  methods  (selected  events  used  to  establish  concepts  or 
patterns),  and 

Deductive  methods  (principles  or  generalizations  applied  to  specific 
Instances). 

3.  Uses  examples  and  non-examples: 

To  define  crlterlal  attributes  and  promote  accurate  general  Izatlcns. 

To  gradually  expand  set  of  examples  to  reflect  real  world. 

To  enrich  concept  with  noncrlterlal  attributes. 

4.  Uses  questions  to  guide  the  search  and  discovery  process;  l.e. ,  uses 
questions: 

To  raise  contradictions. 

To  probe  for  deeper-level  responses. 

To  extend  the  discussion. 

To  pass  responsibility  to  the  class. 

5.  Encourages  students  to  use  examples  and  references  from  their  own 

experiences,  to  seek  clarification  and  draw  parallels  and 

associations  that  aid  understanding  and  retention. 

Relates  Ideas  to  past  learning  and  to  students’  own  spheres  of 
Interests,  concerns,  and  problems. 

6.  Allows  students  to  evaluate  the  appropriateness  of  their  own 

responses  and  then  provides  guidance  as  necessary. 

Provides  cues,  questions  or  hints  as  needed  to  call  attention  to 
Inappropriate  responses. 

7.  Uses  discussion  to  encourage  critical  thinking  and  help  students: 

To  examine  alternatives,  judge  solutions,  make  predictions,  and 
discover  generalizations. 

To  orient,  provide  new  content,  review  and  summarize,  alter  flow  of 
Information,  and  combine  areas  to  promote  the  most  productive  discussion. 
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T£ble_6.  Effective  and  Ineffective  Indicators  for  Engaging  Students  In  the  Learning  Process 
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Rationale  for  the  behaviors  comprising  engagement  in  the  learning  process.  The  following  Is 
a  brief  overview  of  the  rationale  for  each  of  the  subcomponents  pertaining  to  engagement  In  the 
learning  process. 

As  noted  In  Table  6,  effectively  engaging  learners  In  the  learning  process  Involves: 

1.  Eliciting  the  desired  behavior. 
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2.  Providing  opportunities  for  feedback  In  a  non-evaluatlve  atmosphere. 

3.  Using  Individualized  activities  and  supplementary  Materials  as  motivational  aids  (e.g.> 
programed  texts,  simulations,  technical  Illustrations,  desionstratlon  centers). 

4.  Using  meaningful  verbal  praise. 

5.  Monitoring  In-class  assignments  and  checking  for  progress. 

Engagement  In  the  learning  process  begins  by  providing  the  learner  stimulus  material  with 
which  he/she  can  practice  using  the  facts,  action  sequences  or  concepts  being  taught. 
Researchers  generally  agree  that  without  practice,  learning  rarely  If  ever  occurs.  Researchers 
find  that  In  some  learning  contexts  It  Is  not  unusual  to  become  accustomed  to  receiving  lectures 
of  an  hour  or  longer  without  being  provided  an  opportunity  during  the  lecture  to  apply  or  to  use 
what  Is  being  taught.  At  the  college  level,  for  example,  where  considerable  Independence  and 
motivation  are  expected,  the  Instructor  may  appropriately  assume  that  engagement  In  the  learning 
process  will  take  place  at  the  convenience  and  Initiative  of  the  learner  as  one  works  through  the 
lecture  material  In  the  privacy  of  one's  room  or  the  library.  However,  In  other  learning 
contexts  this  assumption  may  not  be  valid  due  to  the  developmental  level  of  the  learner  and/or 
the  mix  of  other  responsibilities  to  which  the  learner  Is  expected  to  devote  time.  This 
possibility  suggests  that  "lecture"  and  practice  should  go  hand  In  hand. 

Levels  of  development.  Independence  and  motivation  generally  are  considered  to  be  different 
for  the  public  school  learner  (and  also  the  military,  vocational  and  technical  student)  than  for 
the  college  student.  It  Is  generally  believed  that  for  the  former,  guided  practice  should  be 
part  of  the  Instruction  Itself,  since  probably  not  all  learners  will  know  how  to  move  from 
lecture  to  practice  without  the  active  and  direct  assistance  of  the  Instructor.  It  Is  often 
suggested,  therefore,  that  each  learner  be  placed  In  a  position  of  grappling  In  trial -and-error 
fashion  with  the  content  being  taught.  It  Is  not  Important  that  the  behavior  be  produced  at  this 
stage  In  a  satisfactory  or  correct  form  but  rather,  that  the  activity  provided  stimulates  an 
attempt  to  produce  the  Intended  behavior.  Such  activity  Is  Intended  to  encourage  the  learner  to 
organize  a  response  which  corresponds  to  the  level  of  behavioral  complexity  being  sought. 
Suggested  ways  In  which  the  desired  behavior  can  be  actively  elicited  from  the  learner  include 
oral  questions,  exercises  from  the  workbook,  specially  prepared  handout  "problems,"  and 
worksheets  administered  by  the  Instructor  during  or  at  the  end  of  a  lesson  which  take  the  learner 
through  the  material  In  a  step-by-step  fashion.  In  this  manner  learners  are  encouraged  to  think 
about,  work  through,  or  otherwise  practice  the  material  In  as  close  a  timeframe  as  possible  to 
the  actual  Instruction  being  provided.  It  Is  generally  believed  that  for  complex  material, 
practice  should  be  Interspersed  throughout  the  lesson,  creating  cycles  of  presentation  and 
practice.  Some  techniques  coownly  recomnended  for  providing  guided  practice  Include: 

1.  Asking  a  large  number  of  questions. 

2.  Guiding  students  In  practicing  the  new  material.  Initially  using  prompts  to  lead  students 
to  the  correct  response  and  later  reducing  them  when  students  are  responding  correctly. 

3.  Checking  for  student  understanding. 

4.  Providing  feedback. 

5.  Correcting  errors. 

6.  Providing  for  a  large  number  of  successful  repetitions. 
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In  addition  to  eliciting  the  desired  behavior  through  guided  practice,  feedback  must  be 
provided  as  to  the  appropriateness  of  the  responses  elicited.  In  order  to  get  and  keep  learners 
responding.  As  crude,  often  Inadequate  responses  are  formed  Into  slightly  less  crude  and  better 
responses,  they,  In  turn,  provide  the  basis  for  eventual  "finely  tuned"  responses  at  a  high  level 
of  accuracy.  Therefore,  researchers  generally  agree  that  the  feedback  being  provided  should  give 
the  learner  unambiguous  Information  as  to  the  adequacy  of  any  response,  but  In  a  manner  that  will 
not  embarrass  or  huelllate  the  learner  or  lower  the  learner's  own  expectations.  Since  Initial 
responses  to  a  guided  practice  exercise  may  be  crude  and  occasionally  Illogical,  the  potential 
for  the  student  to  withdraw  from  or  diminish  his/her  coemltment  to  the  learning  activity  is  at 
Its  greatest  during  this  phase  of  learning.  The  research  suggests  that  the  Instructor's  role  at 
this  stage  Is  not  to  pass  judgment  but  to  provide  feedback  In  a  manner  that  allows  the  learner  to 
Judge  his/her  own  work  using  the  guidance  provided.  Methods  recommended  for  accomplishing  this 
have  Included  having  learners  respond  with  choral  responses,  showing  answers  on  a  transparency 
after  each  problem,  or  simply  orally  supplying  the  answer  at  the  end  of  a  designated  time.  It 
has  been  noted  that  It  may  be  useful  to  the  success  of  these  strategies  to  request  (and 
occasionally  check)  that  learners  write  out  their  responses  In  order  to  be  certain  that  they 
actually  have  engaged  In  the  problem  solution  and  not  simply  waited  for  the  answer  to  be  given. 

Motivation  has  long  been  believed  to  be  an  Important  Ingredient  In  the  learning  process. 
Classroom  researchers  have  noted  Its  Importance  In  engaging  students  In  the  learning  process. 
Researchers  have  noted,  for  example,  that  some  learners  do  not  engage  fully  In  the  learning 
process  even  when  guided  practice  and  non-evaluatlve  feedback  are  provided.  Some  learners  need 
"motivators"  beyond  these  to  become  sufficiently  excited  about  learning  or  energized  to 
continuously  respond  to  the  practice  opportunities  provided.  Motivational  devices  such  as 
performance  Incentives,  programmed  learning  texts,  simulations,  peer  tutoring,  visual 
documentation,  and  demonstration  centers  are  generally  thought  useful  In  engaging  such  students 
In  the  learning  process. 

Foremost  among  these  types  of  motivators  are  Individualized  learning  materials,  since  they 
can  often  be  used  without  Interrupting  regularly  scheduled  Instruction.  It  Is  conmonly 
recognized  that  most  Instructors  do  not  have  the  time  or  flexibility  to  apply  differing 
Instructional  methods  to  different  types  of  learners.  Teaching  even  a  single  group  of  "problem" 
learners  needing  review  or  remedial  help.  In  addition  to  the  "full"  class.  Is  not  a  desirable 
choice  for  most  Instructors.  Curriculum  demands  prescribed  by  the  curriculum  guide  or  text  make 
all  such  dual  teaching  responsibilities  nearly  Impossible  In  most  classrooms.  However,  It  Is 
generally  believed  that  a  library  of  Individualized  remedial  materials  can  allow  for  different 

learning  needs  to  be  met  for  a  heterogeneous  class  of  learners.  When  the  use  of  performance 

incentives,  programmed  texts,  simulations,  and  demonstration  centers  Is  Individualized,  research 
has  suggested  that  some  students  can  be  directed  to  work  Independently  according  to  their  own, 
often  special,  learning  needs  while  others  are  working  on  other  activities.  It  Is  believed  that 
full  engagement  In  the  learning  process  by  all  students  may  require  a  resource  library  of 
individualized  materials  that  can  be  used  with  special  or  poorly  motivated  learners  when  the  need 
arises. 

Another  type  of  motivational  device  thought  to  be  applicable  to  all  learners  Is  the  verbal 
praise  provided  after  a  correct  or  partially  correct  answer.  Research  has  suggested,  however, 
that  verbal  praise  In  the  absence  of  some  kind  of  bona  fide  accomplishment  Is  quickly  seen  by 
learners  as  an  Insincere  response.  Yet,  between  a  glib  and  sometimes  meaningless  response  such 
as  "Correct,"  "OK,"  "Yep,"  and  an  overly  emotional  response  such  as  "That's  perfect,"  or  "Great," 
lies  a  range  of  verbal  rewards  that  may  neither  pass  by  barely  noticed  nor  be  taken  as 

undeserved.  These  verbal  rewards  attempt  to  link  a  learner's  response  with  the  exact  level  of 

accomplishment  attained.  For  example.  Instead  of  simply  informing  the  learner  that  the  response 
was  "Correct,"  the  learner  might  be  notified  that  the  response  was  "Correct  because..."  (e.g., 
the  directions  were  followed  carefully,  the  correct  sequence  of  events  was  chosen,  care  was  taken 
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to  consult  a  reference  book).  Research  has  suggested  that  the  aotlvatlonal  Impact  on  the 
student,  and  the  aeslre  to  keep  engaged  In  the  learning  process.  Is  greater  when  verbal  praise  is 
provided  in  the  context  of  the  operation  successfully  accoaiplished  by  the  student  to  earn  the 
praise.  In  this  manner,  glib  phrases  such  as  "Correct,*  "Good,"  and  "OK"  are  never  worn  out, 

since  they  will  be  associated  with  some  unique  production  by  the  learner;  l.e. ,  the  reason  the 

response  Is  correct,  good,  or  OK. 

The  research  literature  has  suggested  that  meaningful  verbal  praise  Is  especially  Important 
In  the  case  of  partially  correct,  or  correct  but  hesitant  responses.  Here,  although  praise  Is 
important.  It  must  be  tempered  with  a  sign  that  a  better  or  less  hesitant  response  Is  desired. 
To  simply  ignore  the  Inadequacy  or  hesitation  in  favor  of  a  simple  "OK"  Is  to  give  a  sign  to 
others  that  they,  too,  can  provide  a  less  than  adequate  response.  In  these  cases,  the  research 
suggests  that  It  Is  Important  that  the  praise  be  proportional  to  the  adequacy  of  the  response 

(e.g.,  "That's  partly  correct;  now  let's  see  If  you  can  put  It  all  together,"  or  "Good  try;  now 

change  the  first  part  and  you'll  have  It").  Examples  such  as  these  point  out  the  often  fine  line 
between  a  partially  correct  answer  and  a  wrong  answer,  which  is  often  a  matter  of  judgment.  The 
recoonendatlon  from  the  research  here  Is  that  the  learners'  engagement  In  the  learning  process  is 
promoted  more  by  choosing  to  see  that  the  glass  is  "half-full,"  not  "half-empty,"  thereby  leading 
to  praise  with  qualification  for  a  partially  correct  answer. 

Finally,  the  research  literature  has  noted  that  rarely  can  a  response  be  given  that  might  be 
called  "meaningful  verbal  punishment."  Reminders  to  "Study  harder,"  "Pay  greater  attention," 
"Think  some  more,"  or  "Go  over  your  mistakes"  are  often  desirable  stimulants  to  learning. 
However,  phrases  that  ridicule,  demean,  or  draw  the  class'  attention  to  a  learner's  Ineptitude 
rarely.  If  ever,  will  lead  the  learner  to  change  the  behavior  that  led  to  the  careless  or 
Incorrect  response.  In  these  Instances  replacing  an  emotional  response  with  constructive  tips 
for  finding  the  right  answer  Is  commonly  recoonended. 

Another  behavior  for  effectively  engaging  students  In  the  learning  process  Involves 
monitoring  and  checking  during  the  time  students  are  completing  assignments  at  their  seats. 
These  two  processes  go  hand  In  hand,  as  one  Is  thought  not  to  be  effective  without  the  other. 
Monitoring  Involves  systematically  observing  all  aspects  of  the  classroom.  It  also  means  being 
able  to  perform  overlapping  activities;  that  Is,  actively  observing  learners  while  carrying  out 
other  activities,  such  as  speaking  and  demonstrating. 

During  Instructional  activities  such  as  guided  practice  and  Independent  seatwork,  it  Is 
recommended  that  monitoring  occur  within  a  systematic  routine  of  checking.  Rather  than 
monitoring  by  simply  visually  scanning  parts  of  the  classroom,  the  Instructor  should  circulate 
among  students,  checking  responses  In  workbooks  or  exercises.  The  research  literature  Is  careful 
to  point  out,  also,  that  the  Instructor's  behavior  should  not  be  seen  by  learners  as  evaluative 
but  rather,  as  a  helpful  response  to  the  work  being  attempted.  Casual  glances  at  one  paper  and 
then  another,  and  so  on  until  the  entire  room  is  circulated  generally  are  considered  more 

constructive  than  long  Interactions  with  individual  students.  It  has  been  recoanended  that 

Interactions  with  individual  students  be  limited  to  brief  interchanges  that  are  focused  on  a 

particular  problem  the  student  Is  having.  This  cycle  can  be  repeated  many  times.  In  which  case 
the  Instructor  looks  more  closely  the  second  time  around  at  the  work  of  those  learners  who  were 
given  only  a  glance  the  first  time  around.  Monitoring  and  checking  for  progress,  especially 
during  guided  and  Independent  practice,  are  generally  believed  to  represent  two  Important  tools 
for  effectively  engaging  learners  in  the  learning  process. 

Classes  that  exhibit  high  levels  of  engagement  In  the  learning  process  during  In-class 

assignments  have  been  reported  to  have  the  following  characteristics: 

1.  The  instructor  spends  more  time  in  demonstration  (explanation,  discussion)  and  guided 
practice. 
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2.  The  instructor  nukes  sure  students  are  ready  to  work  alone,  by  achieving  a  correct 
response  rate  of  80%  or  higher  during  guided  practice. 

3.  The  seatwork  activity  follows  directly  after  guided  practice. 

4.  The  seatwork  exercises  are  directly  relevant  to  the  demonstration  and  guided  practice 
activities. 

5.  The  Instructor  guides  the  students  through  the  first  few  problems. 


Success  Rate  In  the  Classrocu 


Table  7  summarizes  some  of  the  behaviors  related  to  establishing  a  moderate  to  high  success 
rate. 

The  research  evidence.  The  primary  and  secondary  sources  providing  strong  or  promising 
research  evidence  for  the  subdimensions  of  establishing  a  moderate  to  high  success  rate  are  as 
f ol 1 ows : 


Primary 


Ames,  C.,  t  Ames,  R.  (1973).  Teachers'  attributions  of  responsibility  for  student  success  and 
failure  following  Informational  feedback:  A  field  verification.  Paper  presented  at  the 

annual  meeting  of  the  American  Educational  Research  Association,  New  Orleans,  LA. 

Bennet,  N.t  Oesforges,  C. ,  Cockburn,  A.,  4  Wilkinson,  B.  (1981).  The  quality  of  pupil  learning 
experiences:  Interim  report.  Lancaster,  England:  University  of  Lancaster,  Centre  for 

Educational  Research  and  Development. 

Brophy,  J.,  4  Evertson,  C.  (1976).  Learning  from  teaching:  A  developmental  perspective. 

Boston:  Allyn  and  Bacon. 

Fisher,  C.,  Berliner,  D.,  Fllby,  N.,  Marlave,  R.,  Cohen,  L.,  A  Dlshaw,  M.  (1980).  Teaching 

behaviors,  academic  learning  time  and  student  achievement:  An  overview.  In  C.  Denham  A  A. 
Lleberman  (Eds.),  Time  to  learn.  Washington,  DC:  National  Institute  of  Education. 

Good,  T.,  A  Grouws,  D.  (1979).  The  Missouri  mathematics  effectiveness  project.  Journal  of 
Educational  Psychology,  71  ,  355-362. 

Good,  T.,  Grouws,  0.,  A  Beckerman,  T.  (1978).  Curriculum  pacing:  Some  empirical  data  In 
mathematics.  Journal  of  Curriculum  Studies,  10,  75-81. 

Kounln,  J.  (1970).  Discipline  and  group  management  In  classrooms.  New  York:  Holt,  Rinehart,  and 
Winston. 

Zahorlk,  J.  (1978).  Teacher  verbal  feedback  and  content  development.  Journal  of  Educational 
Research,  63,  419-423. 

Zoyonc,  R.,  4  Brlckman,  P.  (1969).  Expectancy  and  feedback  as  independent  factors  in  task 
performance.  Journal  of  Personality  and  Social  Psychology,  1 1 ,  148-156. 
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Moderate  to  high  rates  of  success  Poor  rates  of  success 

(An  effective  Instructor) _ (An  Ineffective  Instructor) _ (Recogjended  behavior) 

Establishes  a  content  sequence  that  Falls  to  sequence  learning  In  Create  top-down  Instructional 

reflects  prior  learning;  e.g.,  plan-  advance  to  assure  that  all  task-  blocks  In  which  overall  block 
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Varies  the  pace  at  which  stimuli  are  Maintains  same  pace  for  too  long  Use  review,  summarization 

presented  and  continually  builds  a  time,  leading  to  a  monotonous  feedback  and  testing  sessions 

toward  a  climax  or  key  event.  and  static  level  of  Intensity  to  form  Intervals  of  increas- 

and  expectation.  Ing  and  decreasing  intensity 

and  expectation. 


Secondary 


Ausubel,  D.P.  (1963).  The  psychology  of  meaningful  verbal  learning.  New  York:  Grune  and 
Stratton. 

Cronbach,  L.  (1967).  How  can  Instruction  be  adapted  to  Individual  differences?  In  R.  Gagne1 
(Ed.).  Learning  and  Individual  differences.  Columbus,  OH:  Merrill. 

Lundgren.  V.  (1972).  Frame  factors  and  the  teaching  process:  A  contribution  to  curriculum 
theory  and  theory  on  teaching.  Stockholm:  Alngrlst  and  Wlksell. 

Rosenshlne,  B.  (1983).  Teaching  functions  In  Instructional  programs.  The  Elementary  School 
Journal ,  83,  335-351. 

Tobias,  S.  (1982).  When  do  Instructional  methods  make  a  difference?  Educational  Researcher,  11 , 
4-10. 

Rationale  for  behaviors  comprising  a  moderate  to  high  success  rate.  The  following  overview 
provides  the  rationale  for  each  of  the  subcomponents  pertaining  to  success  rate. 

As  noted  In  Table  7,  engaging  your  learners  In  the  learning  process  at  moderate  to  high  rates 
of  success  Involves: 

1.  Establishing  a  content  sequence  that  reflects  prior  learning. 

2.  Correcting  partially  correct,  correct  but  hesitant,  and  incorrect  answers. 

3.  Dividing  Instructional  stimuli  Into  "bite-sized*  chunks  at  the  learners  current  level  of 
functioning. 

4.  Changing  Instructional  stimuli  gradually.  In  small  steps. 

5.  Varying  the  Instructional  tempo  or  pace  to  create  momentum. 

The  first  behavior  for  obtaining  a  moderate  to  high  success  rate  occurs  during  unit  or  block 
planning.  At  that  time,  block  outcomes  are  Identified  and  a  logical  sequence  of  lessons  Is 
chosen  to  achieve  those  outcomes.  Although  the  military  Instructor  rarely  will  have  the 
opportunity  to  design  a  block  of  Instruction,  he  or  she  will  have  or  should  have  the  opportunity 

to  Influence  how  a  block  may  be  1mplemented--and  thereby  effect  meaningful  change.  Researchers 

have  Indicated  that  some  Instructional  blocks  are  planned  with  little  or  no  consideration  given 
to  what  must  come  first  In  a  sequence  of  events  for  the  block  outcome  to  be  achieved.  Although 
some  block  outcomes  may  be  achieved  by  lessons  arranged  In  any  order,  most  cannot.  The  result  of 
poor  lesson  sequencing  Is  often  heard  In  expressions  such  as  "I've  run  out  of  time,"  or  "Had  to 
rush  to  finish  topic  X."  Depending  on  the  circumstances,  these  may  or  may  not  be  excuses  for 
poor  block  planning  but,  more  often  than  not,  they  could  be  avoided  with  good  block  planning. 
Researchers  believe  that  "good  block  planning"  means  arranging  lessons  In  a  sequence  that  works 
toward  long-term  outcomes.  This  is  accomplished  by  making  each  lesson  "dovetail"  or  relate  to 
each  previous  lesson.  This  requires  that  task-relevant  prior  knowledge  be  Imparted  limnedlately 
preceding  or  as  close  In  time  as  possible  to  the  lesson  or  lessons  In  which  It  will  be  needed. 

The  savings  In  time  results  from  either  not  having  to  reteach  forgotten  task-relevant  prior 

learning  that  may  be  necessary  for  the  day's  lesson,  or  not  having  to  reteach  that  day's  lesson 
upon  finding  out  at  some  later  time  that  the  task-relevant  information  needed  for  It  had  not  been 
learned.  Researchers  note  that  a  considerable  amount  of  Instructional  time  during  a  course  of 
Instruction  Is  sometimes  consumed  by  backtracking  to  remedy  deficiencies  in  facts,  skills  or 
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concepts  that  could  have  been  avoided  with  better  planning  at  the  unit  level.  Effective  block 
planning  Mans  uklng  each  lesson  work  to  accomplish  the  goals  of  subsequent  lessons  by  arranging 
a  sequence  of  Instruction  that  builds  to  block  outcoMs  gradually,  In  a  logical  and  systematic 
order.  The  recoaeendatlon  most  comonly  made  here  Is  to  make  each  new  lesson  a  logical  extension 
of  the  previous  lesson  In  order  to  secure  moderate  to  high  success  rates  on  performance  exercises 
during  guided  and  Independent  practice. 

Another  behavior  thought  essential  for  securing  moderate  to  high  success  rates  on  tests, 
homework,  and  problem  sets  Is  the  provision  of  timely  feedback.  During  guided  practice.  In  which 
the  desired  behavior  1$  being  elicited  for  the  first  time,  correctives  should  be  given 
(mediately  after  the  learner's  Initial  response.  The  tlM  between  practice  and  feedback  has 
long  been  considered  In  the  research  literature  as  one  of  the  most  Important  elements  of 
learning.  It  Is  generally  agreed  that  the  longer  that  feedback  Is  delayed,  the  less  likely  It  Is 
to  Influence  the  learner's  performance  on  subsequent  attempts  to  produce  the  behavior.  The 
reasons  provided  for  this  are  that  the  learner  must  hold  a  mental  Image  of  the  first  crude 
response  In  order  for  the  feedback  to  be  effective.  Unfortunately,  mental  Images  do  not  last 
long  after  an  Initial  response  Is  made;  so,  the  effectiveness  of  the  feedback  deteriorates 
rapidly  with  any  delay.  In  order  for  the  learner  to  link  feedback  with  his/her  Image  of  the 
response,  the  corrective  must  follow  Immediately.  Recommendations  for  accomplishing  this  often 
Include  calling  out  the  right  answer  after  each  practice  Item  has  been  completed,  displaying  on  a 
transparency  a  model  for  attaining  a  right  answer,  or  having  learners  check  the  responses  of 
other  learners  according  to  some  standard  provided  by  the  Instructor  or  the  text.  The  research 
notes  also  that  feedback  using  these  and  related  procedures  should  be  administered  In  a 
non-evaluatlve  atmosphere.  Both  Immediacy  and  a  non-evaluatlve  atmosphere  are  recoomiended  In 
order  for  the  feedback  to  have  Its  greatest  effect  on  revising  and  eventually  fine-tuning  the 
response. 

In  order  to  achieve  moderate  to  high  rates  of  success,  the  research  suggests  that  feedback 
should  be  provided  for  each  and  every  Item  given  during  guided  practice.  Missing  the  opportunity 
to  correct  a  wrong,  partially  wrong,  or  even  correct  but  hesitant  response  (which  may  be  correct 
for  the  wrong  reason)  during  this  critical  tlM  can  affect  success  rates  during  later  Independent 
practice  (In  the  form  of  sustained  work  at  desk.  In  workbook  exercises  or  on  homework).  Research 
suggests  that  guided  practice  should  continue  until  a  success  rate  of  approximately  6OT-801  Is 
achieved.  Generally,  that  will  be  possible  only  If  the  correct  responses  and  the  reasons  for 
them  are  provided  after  every  trial  response.  This  moderate  to  high  success  rate  Is  then 
believed  to  pave  the  way  for  still  higher  success  rates  at  the  end  of  Independent  practice,  at 
which  time  about  901  correct  answers  should  be  expected.  The  research  suggests,  however,  this 
higher  rate  of  success  can  rarely  be  achieved  without  first  establishing  a  601  to  801  success 
rate  during  guided  practice.  Finally,  som  research  suggests  that  when  a  601  to  801  success  rate 
is  achieved  during  guided  practice,  feedback  during  independent  practice  may  be  delayed  with  no 
111  effects  on  success  rate,  as  when  correct  answers  are  given  at  the  end  of  the  period  or 
homework  Is  graded  the  next  day. 

The  next  behavior  often  considered  Important  for  establishing  moderate  to  high  success  rates 
Involves  tailoring  block  and  lesson  plans  to  fit  the  learners.  To  be  sure,  blocks  need  to  be 
chosen  to  reflect  some  “whole*  that  Is  comprehensible  to  learners.  But,  the  research  also  points 
out  that  blocks  must  be  sized  to  cover  a  length  of  time  In  which  learners  still  remember  the 
specifics  taught  at  the  beginning  of  the  block  at  the  time  when  broader  concepts  and  abstractions 
are  taught  at  the  end  of  the  block.  Long  blocks  that  ramble  through,  for  example,  many  physical 
laws,  mathematical  procedures,  logistical  operations,  rules,  and  regulations— no  matter  how 
related  the  laws,  procedures,  operations,  rules  and  regulations  may  be— may  prevent  some  types  of 
learners  from  connecting  facts,  rules,  and  sequences  with  the  concepts,  abstractions  and  patterns 
to  which  they  belong.  Research  suggests  that  the  pace  and  size  of  some  blocks  may  have  to  be 
adjusted  for  some  types  of  learners  and  some  types  of  subject  matters. 
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Inappropriate  sizing  at  the  block  level  nas  been  reported  to  create  serious  problems  at  the 
lesson  level.  For  Instance,  when  a  block  topic  Is  too  broad,  the  typical  Instructor  tends  to 
make  lessons  more  abstract  In  order  to  cover  the  block  content  In  the  specified  period  of  time. 
As  a  result  considerable  difficulties  occur  at  the  lesson  level  in  establishing  moderate  to  high 
success  rates.  As  lesson  content  becomes  less  specific,  adequate  periods  of  guided  and 
Independent  practice  are  often  slighted.  There  is  a  rush  to  cover  the  content,  with  perhaps  some 
compromises  made  on  how  we1!  It  Is  covered.  It  has  also  been  found  that  when  this  happens, 
behaviors  at  higher  levels  of  complexity  may  be  taught  without  adequate  attention  to  the 
task-relevant  prior  fac^s,  rules,  and  sequences  that  may  be  necessary  for  attaining  these 
higher-level  outcomes.  It  has  also  been  noted  that  when  Instruction  Is  continually  planned 
around  the  Indirect  Instruction  model  (see  Table  5),  it  may  be  a  sign  that  the  content  area  Is 
too  broad  at  the  block  level  and  that  some  retrenchment  may  be  necessary.  Although  the 
determination  of  block  content  Is  not  the  responsibility  of  the  Instructor,  practical  decisions 
about  the  Implementation  of  this  content.  If  properly  recorded,  can  from  time  to  time  make 
valuable  adjustments  to  a  block,  which  can  be  formally  changed  at  a  later  time.  The  research 
suggests  that  every  effective  Instructional  environment  should  encourage  this  type  of  individual 
Initiative  and  flexibility. 

Finally,  It  has  been  noted  that  the  size  of  the  chunk  chosen  for  lesson  content  must  be  at 
the  learners'  current  level  of  functioning,  If  moderate  to  high  success  rates  are  to  be 
achieved.  When  guided  practice  sessions  or  question  and  answer  sessions  do  not  provide  60S  to 
80S  correct  responses,  lesson  content  may  be  both  too  broad  and  too  complex.  This  Is  commonly 
taken  as  a  sign  that  chunks  of  content  are  not  "bite  sized"  and  that  the  lesson  may  have  to  be 
divided  Into  several  smaller  segments  before  It  can  be  digested  by  learners. 

A  fourth  behavior  for  establishing  moderate  to  high  success  rates  Involves  transitions  within 
and  between  chunks  of  lesson  content.  Although  planning  lesson  content  In  bite-sized  chunks  aids 
learners  In  making  transitions  between  old  and  new  content,  the  research  on  planning  suggests 
this  alone  will  not  be  sufficient  to  assure  moderate  to  high  success  rates.  In  addition,  the 
Instructor  needs  to  present  block  and  lesson  content  In  ways  that  establish  some  overarching 
themes  which  Interconnect  different  parts  of  a  block  or  an  extended  lesson.  Although  not 
mutually  exclusive,  such  methods  Include  the  part -whole,  sequential,  combinatorial,  comparative 
or  hierarchical  approaches  to  organizing  content  that  are  commonly  described  In  teaching  methods 
texts  (Gage  4  Berliner,  1984).  Each  of  these  approaches  represents  a  means  for  organizing 
lessons,  and  content  within  lessons,  that  emphasizes  relationships  among  various  aspects  of  a 
topic.  Some  of  these  approaches  link  content  together  procedural ly,  as  in  the  sequential 
approach,  while  others  link  content  by  emphasizing  themes  and  threads  running  through  the  content 
Itself,  as  In  the  part-whole  approach.  The  research  suggests  that  the  effect  upon  the  learner 
is  to  make  the  transitions  between  parts  of  a  topic  more  comprehensible;  that  Is,  It  helps  the 
learner  see  that  what  Is  being  taught  now  Is  actually  a  part  of  what  has  gone  before  and  what 
will  follow.  This  aids  In  establishing  moderate  to  high  rates  of  success  on  performance 
assessments  Inasmuch  as  the  learner  Is  able  to  build  an  understanding  of  the  topic  gradually  In 
measured  steps,  as  opposed  to  trying  to  put  all  the  bits  and  pieces  together  at  review  and  test 
times.  Easy  and  hard  lessons  may  be  Inevitable,  but  the  research  suggests  that  transitions 
between  them  will  be  more  comprehensible  when  the  common  structure  of  which  they  are  a  part  Is 
made  known  to  learners. 

Another  behavior  for  establishing  moderate  to  high  success  rates  has  been  mentioned 
previously  In  relation  to  several  other  teaching  behaviors.  It  deserves,  however,  special 
mention  again  with  respect  to  the  concept  of  momentum.  This  teaching  effectiveness  Indicator 
adds  a  subtle  but  Important  feature  to  establishing  cycles  of  weekly  and  monthly  review  and 
testing,  as  discussed  earlier  with  regard  to  task  orientation.  To  Illustrate  the  concept  of 
momentum,  some  writers  have  compared  the  practice  of  teaching  with  that  of  conducting  a  symphony 
orchestra.  The  thread  which  binds  these  two  activities  together.  It  is  pointed  out,  is  not 
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content  but  rather  the  simple  fact  that  both  the  maestro  and  Instructor  must  have  a  pattern  or 
theme  that  Is  understood  by  their  respective  audiences  even  if  It  has  never  been  heard  before. 
How  learners  can  be  made  to  understand  In  the  course  of  a  60-minute  lesson  something  they  knew 
nothing  about  before,  and  how  the  conductor  can  make  an  abstract  and  perhaps-never-heard-before 
concerto  recognizable  to  the  audience.  Is  a  topic  that  has  gained  the  attention  of  a  number  of 
writers.  Their  tentative  conclusions  suggest  that  the  explanation  lies  in  the  way  the  Individual 
notes  or  Individual  pieces  of  the  lesson  are  put  together.  How  the  notes  are  played  against  one 
another  Is  reflected  In  the  tune's  melody  and  how  the  Individual  pieces  of  content  are  related  to 
one  another  are  reflected  In  a  lesson's  momentum.  The  literature  on  effective  teaching  generally 
agrees  that  one  of  the  most  Important  Ingredients  for  ensuring  that  a  lesson  will  be  listened  to 
and  understood  Is  to  write  one  that  has  momentum.  Momentum  is  the  varying  pace  at  which  stimuli 
are  presented  In  the  process  of  reaching  a  climax  or  key  event.  In  teaching,  it  Is  suggested 
that  this  be  accomplished  first  by  establishing  cycles  of  weekly  and  monthly  review,  feedback, 
and  testing,  and  then,  by  gradually  Increasing  the  Instructional  pace  and  Intensity  as  the  time 
for  the  major  event  draws  near.  Playing  the  same  Instructional  "note"— or  keeping  the  same 
monotonous  pace  too  long-may  be  as  boring  as  listening  to  a  drab  and  lifeless  musical  score. 
Consequently,  rising  and  falling  actions  are  established  to  set  the  instruction  In  motion  toward 
some  discernible  event.  The  event  can  be  established  by  scheduling  a  cycle  of  review,  feedback 
and  testing  and  by  gradually  Increasing  the  pace  or  tempo  within  the  cycle  until  the  end  of  the 
cycle  is  reached  (e.g.,  a  test  or  laboratory  evaluation),  at  which  time  the  cycle  can  begin  again. 

Some  techniques  commonly  recommended  for  securing  moderate  to  high  success  rates  Include: 

1.  Break  down  the  Instruction  Into  smaller  steps.  Give  the  students  Instruction  and 
practice  on  each  step  before  proceeding  to  the  next  step. 

2.  Provide  the  students  with  explicit  demonstrations  of  skills,  whenever  possible. 

3.  Intersperse  demonstrations  with  questions  In  order  to  maintain  students’  attention  and  to 
check  for  student  understanding. 

4.  Provide  students  with  Instructor-monitored  practice  prior  to  seatwork  activity  so  that 
the  Instructor  can  correct  errors  before  they  become  part  of  the  students'  repertoire. 

5.  With  especially  confusing  material,  provide  pre-corrections  by  advising  the  students 
about  particularly  confusing  areas. 

6.  Provide  sufficient  Independent  practice,  In  terms  of  both  length  and  number  of  exercises, 
to  enable  students  to  master  skills  to  the  point  of  overlearning  (with  additional  exercises  for 
the  slower  students). 

7.  Reteach  material  when  necessary. 


III.  INSTRUCTOR  EVALUATION  PROCEDURES  IN  THE  PUBLIC  SCHOOL  SECTOR 

This  section  briefly  reviews  some  of  the  evaluation  and  feedback  systems  currently  In  use  by 
school  systems  and  State  Departments  of  Education  to  evaluate  and  provide  feedback  to  public 
school  teachers.  The  teaching  effectiveness  Indicators  used  In  one  of  these  systems  are  compared 
with  those  from  the  research  literature  cited  In  the  previous  section. 

Throughout  much  of  the  past  decade,  the  U.S.  has  engaged  In  a  major  movement  of  educational 
reform.  The  primary  Impetus  for  this  movement  has  been  the  release  of  several  reports  critical 
of  the  American  school  system,  such  as  the  often-cited  report  A  Nation  at  Risk,  released  in  1983 
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by  the  National  Coamrlsslon  on  Excellence  In  Education.  The  seeds  of  dissatisfaction  which  In  the 
1960's  and  70's  led  to  competency  testing  requirements  for  students  have  since  burgeoned  Into  a 
mandate  for  comprehensive  educational  reforms  touching  on  a  number  of  new  Initiatives  pertaining 
to  the  evaluation  and  retention  of  both  students  and  their  classroom  teachers.  For  exaaple,  27 
states  to  date  have  passed  major  educational  reform  packages  pertaining  to  required  standards  of 
performance  for  students  and  teachers  (Hardy.  1966). 

Two  related  developments  have  proceeded  from  these  new  Initiatives.  One  has  been  the 
spreading  use  of  teacher  conpetency  testing.  The  second  has  been  an  increase  In  the  creation, 
refinement,  and  use  of  teacher  appraisal  instruments,  which  Include  direct  observation  of 
on-the-job  classroom  performance.  Each  of  these  developments  Is  briefly  reviewed  below. 


Competency  Testing 

Between  1977  and  1985,  the  number  of  states  requiring  some  form  of  teacher  competency  testing 
increased  from  3  states  to  42.  Testing  programs  vary  widely  from  state  to  state,  however, 
differing  In  their  target  populations,  test  content,  and  use  of  results. 

Target  population.  At  least  a  dozen  states  test  students  prior  to  adrisslon  to  teacher 
education  programs.  The  most  common  (and  nearly  universal)  target  population  for  competency 
testing  Is  the  set  of  prospective  teachers  at  or  near  co^>let1on  of  coursework  who  are  attempting 
to  obtain  certification.  Only  three  states  (Texas,  Arkansas,  and  Georgia)  reported  the  use  of 
competency  testing  with  teachers  who  are  already  certified  (Lehmann  A  Phillips,  1987). 

Test  content.  The  most  frequently  employed  competency  tests  measure  knowledge  of  basic 
skills  such  as  reading  and  mathematics.  Hardy  (1966)  reported  that  18  states  require  subject 
area  specialization  tests  (e.g.,  prospective  secondary  science  teachers  are  tested  on  secondary 
science  content).  Additionally,  general  knowledge  of  a  professional  nature  (such  as  teaching 
methodology)  is  tested  by  19  states.  Hardy  pointed  out  that  states  often  employ  a  combination  of 
basic  skills,  subject  area  specialization,  and/or  professional  knowledge  testing.  Two  of  the 
most  popular  competency  tests  currently  In  use  are  the  National  Teacher  Examinations  (Core 
8attery  and  Specialty  Area  Exams),  and  the  Pre-Professional  Skills  Test. 

Test  administration  and  use  of  results.  Competency  tests  are  most  commonly  administered 
either  by  universities  or  by  agents  of  State  Departments  of  Education,  and  are  primarily  used  for 
the  purpose  of  Initial  certification,  rather  than  for  salary  or  promotion  decisions.  Florida  and 
Tennessee  have  utilized  competency  data  in  master  teacher  or  career  ladder  programs,  but  this  Is 
not  the  norm.  Utah  is  currently  allowing  National  Teacher  Examination  scores  as  one  line  of 
optional  evidence  teachers  may  use  In  seeking  promotion.  Competency  data  have  also  been  used  as 
a  means  of  assessing  the  quality  of  teacher  education  programs  In  at  least  eight  states. 

Trends.  The  use  of  competency  testing  as  a  requirement  for  teacher  certification  continues 
to  Increase.  Haney  and  Reldy  (1987)  cited  three  major  initiatives  at  the  national  level  which 
have  appeared  In  only  the  last  3  years  and  which  will  likely  add  to  the  momentum  of  the  teacher 
examination  movement: 

—  As  of  1985,  the  National  Council  for  Accreditation  of  Teacher  Education  (NCATE)  has 
required  teacher  training  Institutions  to  assess  (a)  the  basic  skills  of  candidates  for 
entry  Into  teacher  training,  and  (b)  exiting  student  competencies,  as  a  requirement  for 
eligibility  for  evaluation  for  accreditation. 

—  The  roughly  100  universities  Involved  In  research  and  teacher  training  who  are 
participating  within  the  consortium  known  as  the  Holmes  Group  In  an  effort  to  reform 
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teacher  education.  In  1966  called  for  the  creation  of  Improved  entry  standards  for  the 
teaching  profession.  Although  the  Holmes  Group  Is  ‘seeking  ways  to  broaden  the  nature 
and  scope  of  the  assessment  of  critical  characteristics  and  knowledge  In  the  preparation 
of  a  teacher*  (Cole,  1987,  p.  25),  acknowledgment  Is  made  of  the  need  for  demonstrated 
basic  skills,  subject  area  and  professional  knowledge. 

—  The  Stanford  Teacher  Assessment  Project's  development  of  Innovative  measurement 
devices  Is  In  answer  to  a  need  Identified  by  the  Carnegie  Forum  on  Education  for  a 
"National  Board  of  Teaching  Standards."  The  Project  has  been  funded  for  the  purpose  of 
providing  prototypical  teaching  assessment  devices  for  the  National  hoard. 

As  the  use  of  coo^etency  exams  for  prospective  teachers  spreads,  the  trend  appears  to  be 
toward  the  use  of  more  sophisticated  measures  employing  non-paper-and-pencil  sources  of  data  that 
can  ensure  that  those  entering  the  field  of  teaching  can  actually  demonstrate  their  knowledge  and 
skills  In  the  classroom. 


Performance  Appraisal 


A  second  educational  reform  Initiative  has  led  to  the  Increased  assessment  of  teachers 
already  1"  the  field.  Prior  to  the  past  10  years,  appraisal  policies  for  practicing  teachers 
were  largely  under  the  control  of  the  local  school  districts  In  most  states.  This  continues  to 
be  the  case  In  many  states.  However,  there  Is  a  growing  trend  toward  state-level  Involvement. 
States  such  as  Arkansas,  Florida,  Kentucky,  Oklahoma,  Tennessee,  and  West  Virginia  have  developed 
guidelines  which  local  districts  must  follow  In  developing  teacher  evaluation  Instruments  and/or 
procedures.  Moreover,  some  states  have  mandated  the  use  of  statewide  Instruments  and  procedures 
for  evaluation  of  all  teachers.  Included  among  the  states  which  have  committed  man-hours  and 
money  to  the  development  of  comprehensive,  state-of-the-art  evaluation  systems  for  In-service 
teachers  are  Georgia,  Texas,  Virginia,  North  Carolina,  Mississippi,  and  Delaware. 

In  their  recent  survey  of  the  teacher  evaluation  programs  of  the  nation's  100  largest  school 
districts,  Ellett  and  Garland  (1986)  found  a  significant  qualitative  discrepancy  between  local 
district  systems  and  the  comprehensive  statewide  systems.  Accordingly,  the  remainder  of  this 
section  will  report  first  the  findings  of  Ellett  and  Garland  regarding  the  practices  and  policies 
of  the  large  local  districts  surveyed.  This  will  be  followed  by  a  more  detailed  description  of 
Kentucky's  guidelines  for  teacher  performance  evaluation  (as  an  example  of  a  state  system  which 
provides  such  guidelines),  and  then  by  an  analysis  of  the  statewide  procedures  and  Instruments  In 
use  In  Mississippi  and  Texas. 

Performance  appraisals  by  large  school  districts.  Ellett  and  Garland  requested  Information 
from  the  United  States'  100  largest  school  districts  regarding  the  purposes,  policies,  and 
practices  of  their  teacher  evaluation  systems.  Their  reported  data  represent  a  return  rate  of 
801,  with  no  Information  supplied  regarding  possible  systematic  differences  between  responding 
and  non -responding  districts. 

When  asked  to  rank  order  the  following  four  possible  purposes  of  their  teacher  evaluation 
systems,  respondents'  mean  rankings  reported  "professional  development  for  teachers"  to  be  of 
highest  Importance,  followed  In  descending  order  by  "accountability,"  "personnel  decisions,"  and 
"Instructional  leadership  for  administrators."  Reports  on  the  actual  utilization  of  evaluation 
data  reveal  somewhat  of  a  discrepancy,  however.  The  most  commonly  reported  uses  (selected  by 
over  90*  of  the  respondents)  were  (a)  "development  of  remediation  plans  for  teachers  with 
Identified  deficiencies,"  and  (b)  "teacher  dismissal."  "Promotion/compensation"  and  “merit  pay" 
were  indicated  as  uses  for  appraisal  data  by  50*  and  23.8*  of  the  respondents,  respectively.  In 
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actual  practice,  teacher  evaluation  data  appear  to  be  used  for  both  professional  development  and 
evaluation,  although  almost  twice  as  much  for  the  former  as  for  the  latter  purpose. 

A  formal  written  policy  addressing  teacher  evaluation  was  reported  to  be  In  effect  In  97.5% 
of  the  responding  districts.  Practical  Issues,  such  as  who  Is  evaluated  by  whom  and  when,  are 
Included  In  at  least  90S  of  the  policies;  other  Issues  are  less  frequently  addressed.  "Standards 
for  acceptable  teaching"  and  "orientation  of  evaluators  to  the  evaluation  Instrument"  are 
stipulated  by  70. 7%  and  61. 3%  for  these  policies,  respectively.  Although  91. 3%  reported  that 
evaluation  procedures  are  clearly  written  In  detail,  only  a  fourth  of  the  districts  actually 
require  potential  evaluators  to  demonstrate  mastery  of  the  use  of  the  evaluation  Instrument. 
Finally,  less  than  half  the  districts  reported  policy  provisions  for  "systematic  review  and 
revision  of  teacher  evaluation  policies." 

As  noted  In  Table  8,  the  most  commonly  cited  method  of  evaluating  teachers  was  "direct, 
systematic  observation  of  teachers."  Of  those  districts  enjoying  this  method,  over  701  utilized 
both  a  standardized  observation  form  and  timelines  for  completing  the  observation.  Over  88%  of 
these  districts  reported  the  use  of  post-observation  conferencing  between  the  evaluator  and  the 
teacher.  Rarely  was  the  procedure  for  combining  the  data  from  more  than  one  observation  on  an 
Individual  spelled  out,  with  only  26.6%  reporting  the  existence  of  written  rules  for  such  a 
process. 


Table  8.  Hethods  of  Evaluation  Reported 
by  the  Nation's  100  Largest  School  Districts* 


Methods  _  Percent 


Direct,  Systematic  Observation  of  Teaching  98.8 
Informal  Observation  of  Teachers  85.0 
Peer  Ratings  of  Teacher's  Performance  11.3 
Student  Ratings  of  Teacher's  Performance  3.8 
Student  Achievement  Data  17.5 
Paper-and-PencIl  Examinations  10.0 
Teacher  Self- Evaluation  31.3 
Other  8.8 


aEllett  and  Garland  (1986). 


In  addition  to  the  quantitative  analysis  of  the  survey  data  received,  Ellett  and  Garland  also 
conducted  a  qualitative  analysis  of  the  supporting  documentation  received  from  30  districts. 
Twenty-five  districts  furnished  evaluation  Instruments  and/or  policy  descriptions.  The  structure 
of  the  Instruments  showed  from  10  to  65  performance  criteria  being  listed  under  from  2  to  25 
broad  categories  of  teacher  attitudes.  The  performance  criteria  were  explained  In  detail  or 
clarified  by  behavioral  Indicators  in  less  than  half  of  the  Instruments  received.  Unfortunately, 
although  Ellett  and  Chandler  gave  examples  of  the  broad  categories  employed  ("Preparation  and 
Planning,"  "Classroom  Organization  and  Management,"  and  "Professional  Growth  and  Attitudes"), 
they  gave  no  detailed  Information  regarding  the  specific  criteria  or  behavioral  indicators  found 
on  these  Instruments.  They  did  note,  however,  that  few  of  the  instruments  were  designed  on  the 
basis  of  teacher  effectiveness  research,  and  that  most  Included  assessments  of  high-inference 
variables.  They  stated: 

...many  school  districts'  teacher  evaluation  Instruments  depend  totally  upon  personal 
judgments  of  evaluators  who  receive  little  or  no  systematic  training  ...  relatively  few 
criteria  were  at  a  level  of  behavioral  specificity  necessary  to  train  observers  to  make 
reliable  assessments  of  teacher  performance  ...  It  Is  apparent  that  translation  of 
state-of-the-art  practices  In  teacher  evaluation  from  large-scale  assessment 
applications  to  local  school  district  uses  seems  sorely  lacking,  (p.  26) 
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Although  Eliott  and  Garland's  analysis  of  tho  state  of  the  art  of  teacher  evaluation  at  the 
school  district  level  awy  be  overly  critical,  as  the  following  data  will  suggest.  It  probably 
does  accurately  describe  conditions  within  at  least  some  school  districts. 

Performance  appraisals  by  State  Departments  of  Education.  The  development  of  statewide 
guidelines  or  mandates  for  teacher  evaluation  In  the  U.S.  has  been  Intimately  tied  to  the 
creation,  by  many  states,  of  teacher  Incentive  programs,  such  as  career  ladder  programs  or 
bonuses  for  outstanding  performance.  Of  the  12  states  previously  Identified  as  having  statewide 
guidelines  and/or  Instruments,  all  but  Oklahoma  have  state  Incentive  programs  which  are  either 
fully  Implemented,  In  the  pilot-testing  stage,  or  otherwise  In  the  process  of  development.  A 
recent  count  (Cornett,  1986)  placed  the  number  of  states  Involved  to  some  extent  In  Incentive 
programs  at  29.  The  evolution  of  Incentive  systems  has  had  significant  Implications  for  teacher 
evaluation.  In  an  update  on  trends  In  teacher  Incentive  programs,  Cornett  reported  that  states 
with  Incentive  programs  are  departing  from  traditional,  relatively  subjective,  global  ratings  by 
principals.  Emerging  program  characteristics  cited  Include  research-based  Instrumentation,  the 
Inclusion  of  multiple  sources  of  evaluation  data,  and  greater  emphasis  on  evaluator  training. 
Three  specific  state  evaluation  systems  are  considered  below: 

Kentucky.  In  March  1985,  the  Kentucky  Board  of  Education  Issued  Its  Guidelines  for  Teacher/ 
A<ka1n1strator  Performance  Evaluation  In  Kentucky,  as  required  by  Kentucky  Senate  Bill  364  enacted 
In  1984.  The  Guidelines,  as  stipulated  by  legislation,  require  (a)  that  evaluation  procedures  be 
developed  by  certified  teachers  and  administrators,  (b)  that  evaluation  Include  an 
evaluator/evaluatee  conference,  (c)  that  evaluators  be  trained  In  evaluation  techniques  both 
general  and  specific  to  the  local  system,  and  (d)  that  the  procedures  developed  Include  plans  to 
aid  each  evaluatee  In  increasing  his  or  her  effectiveness.  All  evaluators  are  trained  and  tested 
in  4-day  sessions. 

The  Guidelines  require  that  the  following  performance  criteria  be  Included  In  the  evaluation 
Instrument.  (Others  may  be  added  at  the  local  school  district's  discretion.) 

1.  Performs  professional  responsibilities  and  duties  as  outlined  In  the  job  description, 
with  regular  attendance  and  punctuality. 

2.  Uses  Instructional  strategies  and  processes  effectively. 

3.  Demonstrates  effective  Interpersonal  and  communication  skills  with  peers,  subordinates, 
students  and  parents. 

4.  Plans  and  evaluates  Instructional  activities. 

Also  required  are  bahavloral  indicators  which  define  the  criteria.  Each  Indicator  Is  to  be 
written  such  that  acceptable  quality  of  performance  Is  specified.  The  Guidelines  Include 
examples  of  matched  criteria  and  Indicators,  as  supplied  by  Dr.  Jim  Sweeney,  Iowa  State 
University.  These  may  be  found  In  Appendix  A  (specimen  1).  Also  Included  In  the  Guidelines  are 
the  teacher  evaluation  Instruments  of  Danville  and  Covington,  Kentucky.  On  these  forms,  each 
Indicator  consists  of  a  range  of  observed  behaviors  which  are  assigned  ratings.  The  Danville 
form  lists  performances  for  each  Indicator  which  may  be  rated  unsatisfactory.  Improvement  needed, 
competent,  very  good,  or  outstanding.  The  Covington  Instrument  assigns  3,  2,  1,  or  0  points  to 
each  level  of  performance.  The  summary  forms  for  recording  evaluation  data  from  an  observation 
in  Oanvtlle  and  Covington  are  also  provided  In  Appendix  A  (specimens  2  and  3). 

Mississippi .  Use  of  the  Mississippi  Teacher  Assessment  Instruments  was  fully  Implemented  In 
the  1986-1987  school  year.  Results  of  beginning  teachers'  performance  evaluations  are  to  be  used 
In  state  approval  decisions  concerning  teacher  training  programs.  Additionally,  results  of 
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all  teachars1  performance  evaluations  will  determine  each  teacher's  eligibility  for  a  $1,000 
Incentive  bonus  to  be  awarded  In  1987-1968.  Documentation  from  the  Bureau  of  School  Improvement 
Indicates  the  use  of  three  evaluators:  one  external,  the  building  principal,  and  a  peer 
teacher.  This  policy  Is  targeted  at  beginning  teachers.  Cornett  (1986)  reported,  however,  that 
for  Incentive  bonus  purposes,  only  trained  administrators  will  act  as  evaluators.  Provisions 
have  been  made  for  remedial  training  of  teachers  who  do  not  qualify  for  the  merit  pay. 

The  three  Instruments  which  comprise  the  Mississippi  Teacher  Assessment  Instruments  measure  a 
total  of  14  "competencies"  which  are  defined  by  42  Indicators.  Appendix  A  (specimen  4)  provides 
a  list  of  these  competencies  and  Indicators.  Indicators  are  rated  from  1  to  5,  with  detailed 
descriptors  provided  to  clarify  the  meaning  of  each  scale  point.  An  example  of  such  descriptor 
Information  Is  Included  with  specimen  4  In  Appendix  A.  Competencies  I  to  IV  are  assessed  with 
the  Teaching  Plans  and  Materials  (TPM)  Instrument.  Information  to  be  used  by  the  evaluator(s)  In 
completing  the  TPM  comes  from  a  portfolio  and  a  questionnaire,  both  completed  by  the  evaluatee. 
In  contrast,  the  evaluator  Is  to  use  on-the-job  observation  in  the  completion  of  the  Position 
Skills  and  the  Interpersonal  Skills  Instruments. 

Texas.  Texas  legislation  In  1984  established  a  career  ladder  Incentive  system  for  teachers 
and  required  the  development  of  a  statewide  evaluation  plan  and  criteria.  In  1984-1985  and  In 
1985-1986,  the  career  ladder  employed  locally  developed  evaluation  Instruments.  The  statewide 
plan  was  fully  Implemented  In  1986-1987,  after  being  pilot-tested  In  six  districts  the  preceding 
year. 


The  Texas  Teacher  Appraisal  System  (TTAS)  policies  are  comprehensive.  Title  19,  Part  II  of 
the  Texas  Administrative  Code  and  Statutory  Citations,  addresses  In  detail  Issues  of  appraiser 
qualifications,  teacher  orientation,  observation  and  conferencing  processes,  teacher  response  and 
appeals,  professional  growth  plans,  teacher  self-appraisal,  summatlve  appraisal,  and  scoring 
procedures.  Notable  points  Include  requirements  for  a  minimum  of  six  program-orientation  hours 
for  teachers,  required  pre-  and  post-conferences,  and  written  quantitative  procedures  for 
combining  the  results  of  multiple  observations  and  for  assigning  qualitative  rankings  to 
sumnatlve  scores. 

The  Teacher  Orientation  Manual  describes  the  process  of  literature  review  and  teacher  Input 
which  were  part  of  the  development  of  the  TTAS  criteria.  The  manual's  appendix  lists  the 
relevant  research  references.  The  TTAS  was  Intended  to  have  a  narrow  focus  on  criteria  which 
could  be  validly  and  reliably  measured,  and  which  were  Identified  by  research  as  being 
significant. 

The  four  classroom  teaching  domains  thus  Identified  were  the  following: 

1.  Instructional  Strategies 

2.  Classroom  Management  and  Organization 

3.  Presentation  of  Subject  Matter 

4.  Learning  Environment 

An  additional  domain.  Growth  and  Responsibilities,  Is  included  for  which  full  credit  Is  given  in 

the  absence  of  any  documentation  of  unacceptable  practices.  Local  districts  may  develop 

additional  domains  and  criteria,  but  results  of  these  will  not  Impact  career  ladder  decisions. 
Appendix  A  (specimen  5)  Includes  the  observation  record  which  allows  indicators  to  be  scored 

directly,  as  well  as  the  appraisal  record  on  which  are  documented  the  domain  scores  from  several 

observations  and  which  shows  the  calculation  of  the  overall  summary  performance  score. 
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Finally,  It  can  be  noted  that  when  the  Indicators  of  teaching  effectiveness  fro*  the  research 
literature  reviewed  In  Section  II  of  this  report  are  compared  with  those  Indicators  listed  on  the 
Texas  Teacher  Appraisal  Instrument,  a  791  correspondence  occurs.  That  Is,  791  or  about  22  out  of 
the  28  Indicators  of  teaching  effectiveness  Identified  In  Tables  1,  2,  3,  6,  and  7  of  Section  II 
have  related  or  corresponding  Items  on  the  Texas  Teacher  Appraisal  Instrument.  Conversely,  only 
approximately  581  of  the  teaching  effectiveness  indicators  on  the  Texas  instrument  have 
corresponding  or  related  entries  In  Tables  1,  2,  3,  4,  and  7,  Indicating  the  Inclusion  on  this 
Instrument  of  both  research-based  and  a  large  number  of  professionally  based  behaviors  typical  of 
appraisal  devices  In  the  public  school  sector. 


IV.  DESCRIPTION  OF  INSTRUCTOR  EVALUATION  FORMS  IN  THE  AIR  FORCE, 

ARMY,  AND  NAVY  AND  HOW  THEY  ARE  USED  IN  THEIR  INSTRUCTIONAL  ENVIRONMENTS 

This  section  reviews  the  current  Instructor  evaluation  procedures  used  by  the  Air  Training 
Comand  (ATCR  52-8)  and  compares  the  teaching  effectiveness  Indicators  on  ATC  Form  281  with  those 
derived  from  the  research  literature.  This  section  also  briefly  summarizes  the  Instructor 
evaluation  and  feedback  procedures  currently  In  use  by  the  Army  and  Navy,  with  emphasis  on 
commonalities  among  the  three  services. 


Instructor  Evaluation  in  the  Air  Force 


The  evaluation  of  Instructors  In  technical  training  centers  In  the  Air  Force  Is  comprised  of 
five  distinct  types  of  data  and  related  procedures: 

1.  ATC  Form  281,  Instructor  Evaluation 

2.  ATC  Form  736,  Student  Critique 

3.  Training  Evaluation  Report 

4.  Student  Exercises  and  Tests 

5.  Informal  Peer  Evaluation 

ATC  Form  281  :  Instructor  Evaluation.  This  form  contains  35  items,  divided  among  12  general 
factors.  The  12  general  factors  measured  are: 

Preparation  {4  items) 

Content/Structure  of  Presentation  (3  Items) 

Presentation  Skills  (3  Items) 

Caenunlcatlon  Skills  (3  Items) 

Demonstration  Skills  (2  items) 

Use  of  Training  Aids/Materials  (2  Items) 

Question/Answer  Techniques  (4  Items) 

Management  (5  Items) 

Supervising  Group  Activity  Skills  (2  items) 

Measurement  (2  items) 

Personal  Qualities  (3  Items) 

Other  (2  items,  with  an  option  to  add  others) 

Each  Item  on  Form  281  Is  rated  on  a  4-polnt  scale  as  either  Outstanding,  Satisfactory ,  Needs 
Improvement,  or  Not  Applicable.  Space  is  provided  in  the  right-hand  margin  of  each  item  for 


40 


comments.  Raters  are  advised  on  the  top  of  the  form  that  an  "Outstanding"  or  “Needs  Improvement" 
rating  for  any  item  requires  justification  in  the  comment  space  provided  and  that,  additionally, 
a  "Needs  Improvement"  rating  must  be  followed  up  with  a  subsequent  evaluation  within  30  days.  In 
addition  to  the  35  item  scores,  a  single  overall  performance  rating  is  given,  using  the  scale 
points  of  "Outstanding"  (when  more  than  75S  of  the  Items  are  rated  "Outstanding*  and  no  Item  Is 
rated  "Needs  Improvement");  Satisfactory  (when  the  majority  of  Items  are  rated  either 
“Satisfactory"  or  "Outstanding"  and  less  than  25*  are  rated  "Needs  Improvement");  "Needs 
Improvement"  (when  more  than  50*  of  the  Items  are  rated  "Needs  Improvement").  Space  Is  also 
provided  for  recomnendatlons  for  Improvement  and/or  general  conaents  and  for  coammnts  pertaining 
to  a  follow-up  evaluation,  should  any  Item  be  rated  as  needing  improvement. 

The  Instructor' s  designated  supervisor  completes  Form  281  after  observing  the  instructor 
presenting  a  typical  lesson  for  a  minimum  of  50  minutes  In  the  classroom,  the  laboratory  or  the 
field.  New  Instructors  must  be  evaluated  every  30  days  for  3  months  with  Form  281  and,  then, 
once  each  quarter  until  the  Instructor  attains  the  designation  of  master  Instructor,  at  which 
time  evaluation  Is  yearly.  The  results  of  Form  281  are  conveyed  to  the  Instructor  In  a 
conference  setting  shortly  after  the  evaluation,  at  which  time  It  Is  customary  for  the  supervisor 
to  review  the  Instructor's  strengths  and  weaknesses,  with  particular  reference  to  Items  checked 
"Outstanding"  or  “Needs  Improvement"  and  the  comments  which  have  been  made  to  justify  these 
ratings.  Visits  to  ATC  classrooms  and  post-instruction  Interviews  with  Instructors  at  Lackland 
AFB,  Keesler  AFB,  Lowry  AFB,  and  Sheppard  AFB  by  the  contractor  during  the  months  of  June  and 
July,  1987,  suggest  that:2 

.  Overall,  Instructors  tend  to  value  the  feedback  provided  them  by  Form  281. 

.  Some  feel  a  need  for  more  specific  feedback,  particularly  with  respect  to  Items  on 

which  they  have  been  rated  "Satisfactory,"  but  for  which  they  wish  to  attain  a  rating 
of  "Outstanding." 

.  Little  variation  In  ratings  occurs  either  within  or  between  Individuals;  most 

•  Instructors  receive  a  rating  of  "Satisfactory"  on  most  Items. 

.  Generally,  Instructors  believe  Form  281  Is  more  appropriate  for  formal  classroom 
Instruction  than  for  laboratory  work,  performance  classes,  or  field  exercises,  which 
tend  to  require  Instructional  behaviors  more  specific  to  these  Instructional  contexts. 

.  Most  Instructors  would  like  to  receive  more  written  support,  recommendations  and 

justifications  in  the  coonents  section  of  Form  281,  and  most  Instructors  would  like  to 

see  more  space  allotted  to  comments  on  the  form. 

ATC  Form  281  appears  In  Appendix  C. 

ATC  Form  736:  Student  Critique.  This  Is  a  single-page,  open-ended  form  for  students  who 
wish  to  voluntarily  evaluate  the  training  course  and/or  Instructor.  Students  are  Instructed  to 
use  this  form  to  evaluate  their  training,  student  group  or  base  support  facilities  and  services, 
but  to  use  a  separate  Form  736  for  each  of  the  areas  being  evaluated.  Additionally,  Form  736 
Includes  space  for  the  course  title  and  number,  period  of  training,  focus  of  the  evaluation 
(training,  school  squadron,  or  base  support  facilities),  and  type  of  critique  (Individual,  group 
or  oral -type).  The  student  may  request  to  receive  the  results  of  his  critique.  In  which  case  a 
check  Is  placed  In  an  appropriate  box.  Form  736  also  Includes  a  space  for  the  Instructor  to 
Identify  any  follow-up  action  taken  and  the  method  of  contacting  the  student  when  the  results  of 


2See  Appendix  8  for  the  specific  ATC  classrooms  observed  by  the  contractor  at  these  bases. 
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the  critique  are  requested  by  the  student.  Form  736  may  be  submitted  Individually  or  a  single 
Form  736  may  be  submitted  by  an  entire  class.  In  the  case  of  the  latter  (required  by  some 
schools  and  Instructors),  a  predesignated  student  solicits  oral  comnents  from  the  class  during  a 
specially  allotted  time  In  which  the  Instructor  Is  absent.  Generally,  coments  made  by 
Individual  students  with  which  50X  or  more  of  the  class  agrees  are  recorded  anonymously  on  Form 
736.  Visits  to  ATC  classrooms  and  post-instructional  Interviews  at  Lackland  AFB,  Keesler  AFB, 
Lowry  AFB,  and  Sheppard  AFB  suggest  that: 

1.  Form  736  tends  to  be  most  valued  by  Instructors  for,  and  Is  most  frequently  used  by 
students  for,  Identifying  specific,  well-defined  weaknesses  of  a  course  or  Instructor.  Form  736 
tends  to  be  less  valued  by  Instructors  for,  and  Is  less  frequently  used  by  students  for, 
documenting  good  or  exceptional  teaching  performance.  In  which  case  student  comments  tend  to  be 
congratulatory  and  Instructional 1y  vague. 

2.  Instructors  tend  to  treat  student  requests  for  feedback  on  Form  736  seriously,  frequently 
getting  In  touch  with  Individual  students  to  discuss  complaints,  when  possible. 

3.  Overall,  Form  736  Is  most  used  when  Identifiable  problems  persist  In  the  classroom  and 
students  believe  there  1$  a  willingness  on  the  part  of  the  Instructor  to  accept  feedback.  In 
classrooms  where  this  atmosphere  Is  not  communicated.  Form  736  tends  not  to  be  used  by  Individual 
students. 

4.  The  number  of  Individually  submitted  736's  varies  considerably  from  Instructor  to 

Instructor,  with  some  senior  Instructors  reporting  never  having  received  one,  and  other 

Instructors  reporting  having  received  some  each  time  their  course  was  taught.  Generally,  the 
number  of  Individually  submitted  Form  736's  appears  not  to  be  large. 

Form  736  appears  In  Appendix  C. 

Training  Evaluation  Report.  The  training  evaluation  report  Is  a  post-instruction 

questionnaire  which  queries  graduates  on  the  Job  about  the  adequacy  of  their  course  Instruction. 
4  small  set  of  graduates  of  each  training  school  are  randomly  sent  a  questionnaire  approximately 
6  months  after  training,  while  performing  the  duties  related  to  their  occupational  specialty. 
The  questionnaire  provides  students  with  an  opportunity  to  conment  on  the  adequacy  of  the 
training  received  from  Individual  courses  as  well  as  Instructors.  Most  Instructors  and  some 
supervisors,  however,  cannot  remember  receiving  results  from  the  questionnaire  and,  hence,  it 
generally  Is  not  considered  an  evaluative  or  diagnostic  tool  for  Instructor  feedback.  From  the 
Instructors'  perspective,  most  of  this  Information  remains  at  the  conmand  level  for 

administrative  purposes. 

Student  Exercises  and  Tests.  These  Include  both  Informal  exercises  and  formal  tests  of 
student  knowledge  given  In  the  context  of  Instruction.  Student  exercises  include  practice 
exercises,  routinely  a  part  of  most  lessons,  whereas  tests  Include  both  the  results  of 
performance  evaluations  that  are  graded  within  a  course  and  formal  tests  at  the  completion  of  the 
course.  Typically,  Instructors  report  using  the  results  of  these  exercises  and  tests  for 
self-evaluation  and,  when  necessary,  to  revise  Instructional  procedures,  although  on  an  Informal 
basis.  However,  Instructors  report  that  student  data  In  the  form  of  responses  on  exercises  and 
grades  on  tests,  although  reported  to  be  useful  as  general  benchmarks  of  teaching  performance,  do 
not  provide  specific  diagnostic  information  as  to  what  teaching  behaviors  might  be  related  to  a 
good,  mediocre,  or  poor  class  performance.  Therefore,  the  Informal  use  of  these  data  for 
instructor  evaluation  and  their  lack  of  diagnostic  specificity  with  regard  to  specific  Instructor 
behaviors  appear  to  limit  their  usefulness  for  feedback  enhancement. 
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Informal  Peer  Evaluation.  A  final  source  of  data  for  the  evaluation  of  training  Is  provided 
by  peer  evaluations.  In  which  Instructors  voluntarily  observe  each  other.  The  frequency  of  peer 
evaluation  varies  froa  school  to  school,  with  most  schools  Including  some  peer  evaluation  and  a 
few  schools  scheduling  frequent  evaluations  as  a  standard  practice.  Peer  evaluations  most 
commonly  use  Form  281,  which  either  Is  completed  by  the  peer  observer,  just  as  It  would  be 
completed  by  the  supervisor,  or  Is  used  as  a  general  guide  for  what  to  observe.  Feedback  Is 
Informal,  many  times  taking  the  form  of  brief  chats  between  Instructors  Immediately  after  the 
observation  or  queries  In  the  breakroom  as  to  "What  did  you  see?"  Visits  to  a  sample  of  ATC 
classrooms  at  Lackland  AFB,  Keesler  AFB,  Lowry  AFB,  and  Sheppard  AFB  suggest  that: 

1.  Peer  evaluation  Is  valued  by  Instructors  as  one  of  the  most  useful  sources  of  feedback 
they  receive. 

2.  Peer  evaluation  often  occurs  too  Infrequently,  due  to  time  and  scheduling  coimnltments. 

3.  Peer  evaluations  tend  to  be  the  most  helpful  when  the  peer  evaluator  looks  for 
Instructional  effectiveness  In  ways  that  supplement  or  enhance  Items  on  Form  281;  l.e.,  makes 
extensive  comments  In  the  space  provided. 

4.  Although  Form  281  often  serves  as  a  starting  reference  for  a  peer  evaluator,  observers 
often  add  to  It  using  evaluative  criteria  of  Intuition,  professional  judgment,  and  personal 
experience.  Often  such  additions  appear  In  the  comments  portion  of  Form  281. 


Comparison  of  Items  on  ATC  Form  281  with  Indicators  of  Teaching  Effectiveness 
from  the  Research  Literature 


In  this  section  the  Items  on  Form  281  are  compared  with  the  Indicators  of  teaching 
effectiveness  reviewed  In  Section  II  of  this  paper.  In  that  section  five  key  behaviors  (Clarity, 
Variety,  Task  Orientation,  Engagement  In  the  Learning  Process,  and  Moderate  to  High  Rates  of 
Success)  were  Identified  from  the  research  literature.  These  five  key  behaviors.  In  tur”. 
yielded  28  different  subdimensions  of  teaching  effectiveness  that  were  Identified  In  Tables  1,  2, 
3,  6,  and  7.  In  Table  9,  the  35  Items  on  Form  281  are  "mapped  onto"  these  28  behaviors  In  an 
effort  to  determine  the  degree  to  which  current  Air  Force  Instructor  evaluation  criteria  reflect 
the  criteria  for  teaching  effectiveness  commonly  cited  In  the  research  literature. 

Of  the  35  behaviors  on  ATC  Form  281  ,  28  Items  or  80S  have  some  degree  of  correspondence  with 
the  Indicators  from  the  teaching  effectiveness  literature.  Those  ATC  Form  281  Items  for  which  no 
counterpart  could  be  found  were: 

A2  Classroom  neat  and  orderly.  Seating  arrangement  appropriate.  Items  Identified 
needing  repair,  such  as  burned  out  lights. 

01  Correct  enunciation  and  granmtar  used. 

H4  Instructor  properly  used  the  MIR  Instructor. 

K2  Instructor's  professionalism  set  the  proper  example  for  bearing,  behavior,  and  dress. 

K3  Positive  rapport  with  students  established. 

LI  Students  In  compliance  with  appropriate  dress  and  appearance. 

L2  Importance  of  safety  emphasized  and  compliance  with  safety  standards  ensured. 
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Table  9.  Comparison  of  Key  Indicators  of  Teaching  Effectiveness  froa  the  Research  Literature 

with  I  teas  on  ATC  Fora  281 
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nterna 1  sumary  given  if  lesson  not  ended). 


Key  Teaching  effectiveness  Indicators  fro*  Related  Items  on  Fora  281 

behavior _ research  literature  * 
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Uses  learner  Ideas  and  participation  to  E.  2  Students  Involved  In  demonstration.  If 

foster  lesson  objectives  when  appropriate  appropriate. 

(weekly).  H.  5  Student  participation  encouraged. 

I.  2  Group  members  encouraged  to  participate. 


Key  Teaching  effectiveness  Indicators  froa  Related  Items  on  Form  281 

behavior _ research  literature _ 

Task  1.  Develops  unit  and  lesson  plans  In  B.  2  All  objectives  covered.  Lesson  flowed 


Monitors  seatwork  by  circulating  and  J.  1  Progress  check  and  perforaance/wrltten 

frequently  checking  progress,  tests  administered  properly. 


Key  Teaching  effectiveness  Indicators  froa  Related  Iteas  on  Fora  281 

behavior _ research  literature 
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As  can  be  noted,  several  of  the  above  Itens  are  specific  to  the  military  classroom  and, 
therefore,  would  not  be  expected  to  have  counterparts  In  the  teaching  effectiveness  research 
conducted  In  public  school  classrooms.  Conversely,  It  can  be  noted  from  Table  9  that  9  of  the 
28— or  approximately  one-third  of  the  teaching  dimensions  identified  In  the  teaching 
effectiveness  1 1terature— do  not  directly  correspond  with  any  Item  on  Form  281,  suggesting  the 
possibility  of  expanding  the  content  of  Form  281  to  reflect  these  behaviors,  where  applicable  to 
the  ATC  classroom. 


Instructor  Evaluation  In  the  Army  and  Mavy  and  Some  Comparisons  with  Form  281 


This  section  briefly  reviews  the  Instructor  evaluation  forms  used  by  the  Army  and  Navy. 

Army.  Simlllar  to  the  Air  Force,  the  Army  uses  a  single-page,  two-sided  form  for  Instructor 
evaluation.  Five  general  teaching  effectiveness  factors  are  measured  on  this  form,  comprising  a 
total  of  25  Items,  as  compared  to  35  Items  on  ATC  281.  These  Items  are  distributed  across  five 
basic  factors,  as  compared  to  12  factors  on  ATC  Form  281.  These  factors  are: 

Introduction/Statement  of  Objectives  (3  Items) 

Lesson  Presentation  (6  Items) 

Instructor  Comnunlcatlon  Skills  (6  Items) 

Instructor  Preparation  (7  Items) 

Training  Site  Management  (3  Items) 

Each  of  these  factors  Is  similar  to  one  or  more  of  the  12  general  factors  measured  on  ATC 
Form  281.  Items  on  the  Army  form,  however,  employ  a  7-polnt  scale,  as  conpared  to  a  4-point 
scale  on  ATC  Form  281,  and  representing  the  following  intervals. 

NR  -  Not  Relevant 

1  *  Falling 

2  ■  Poor 

3  ■  Unsatisfactory 

4  *  Marginally  Satisfactory 

5  3  Fully  Satisfactory 

6  •*  Outstanding 

An  additional  difference  between  the  Army  form  and  ATC  Form  281  Is  that  each  item  on  the  Army 
form  is  assigned  a  factor  weight  ranging  from  1  to  6,  depending  on  Its  perceived  importance  to 
Army  Instruction.  Hence,  an  Item  rated  "6"  and  assigned  a  weight  of  "5"  would  be  scored  as 
"30.“  The  maximum  score  attainable  across  all  items,  if  each  Item  were  rated  “6"  and  multiplied 
by  Its  proper  weight.  Is  600.  This  base  Is  then  adjusted  downward  for  the  number  of  nonrated 
items  by  multiplying  the  total  number  of  nonrated  items  by  6  and  subtracting  this  value  from  the 
base.  This  adjusted  base  Is  then  divided  Into  the  weighted  sum  of  all  the  Items  and  multiplied 
by  100  to  arrive  at  a  percentage  of  the  total  points  possible.  The  Instructor  is  then  assigned 
an  overall  rating  based  on  the  following  ranges  of  percentages: 


0  -  66 

Below  Average 

67  -  83 

Average 

84  -  94 

Superior 

95  -  100 

Exceptional 

Space  is  provided 

at  the  end  of 

section.  Space  is  also  provided  at  the  end  of  all  sections  for  general  comnents.  A  critique  and 
feedback  session  with  the  instructor  is  required  no  later  than  48  hours  after  the  assessment.  A 
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matching  of  Items  on  the  Anqy  Instructor  evaluation  form  with  the  teaching  effectiveness 
Indicators  presented  In  Tables  1,  2,  3,  6,  and  7  reveals  that  64X  of  the  Items  have  some  degree 
of  correspondence  with  the  teaching  effectiveness  Indicators  commonly  referenced  In  the 
literature,  as  compared  to  80S  for  ATC  Form  281.  As  In  the  case  of  Form  281,  5  of  the  25  Items 
on  the  Army  Instructor  evaluation  form  appear  to  be  military-specific  and,  therefore,  would  not 

be  expected  to  have  counterparts  In  the  teaching  effectiveness  research  conducted  In  public 

school  classrooms. 

A  copy  of  the  Arnjy  evaluation  form  appears  In  Appendix  D. 

Navy.  Although  the  Air  Force  and  Army  Instructor  evaluation  procedures  are  somewhat  similar 
In  approach,  the  Navy  Instructor  evaluation  procedure  differs  considerably  from  both  In  scope  and 
depth.  The  Navy  approach  to  evaluation  Is  referred  to  as  the  Course  Evaluation  System  (CES), 
emphasizing  Its  focus  on  the  evaluation  of  the  organization  of  the  subject  matter  being  taught 
(the  course),  as  well  as  the  presentation  or  delivery  of  the  subject  matter  by  an  instructor.  In 
the  Navy  system,  the  evaluation  of  the  Individual  instructor  Is  only  a  part  of  this  more  general 
evaluation  system. 

The  CES  Is  divided  Into  three  distinct  parts,  any  one  of  which  may  be  completed  to  the 

exclusion  of  the  others.  Also,  due  to  the  labor-intensive  nature  of  the  CES,  use  of  the  system 
Is  recommended  only  If  a  course  or  Instructor  Is  shown  to  be  having  “serious  problems,"  such  as 
negative  feedback  from  superiors,  high  attrition,  critical  student  cooments,  poor  student 
performance  on  tests,  negative  instructor  review,  or  an  unusual  number  of  setbacks.  The  three 
major  components  of  the  CES  are: 

Objective  Classification 

Objective  and  Test  Adequacy  and  Consistency  Evaluation 

Presentation  Evaluation 

The  first  two  components  will  be  summarized  briefly,  as  only  the  third  component. 
Presentation  Evaluation,  Involves  the  direct  evaluation  of  Instructors. 

Objective  Classification.  For  this  operation,  long-term  or  terminal  course  objectives  are 
subdivided  Into  enabling  objectives  and  classified  according  to  (a)  whether  the  objective 
requires  the  student  to  remember  or  use  Information;  (b)  If  "use,"  whether  use  should  be  aided 
(for  example,  with  diagrams)  or  unaided;  and  (c)  whether  or  not  the  objective  requires  the 
transfer  of  learning  to  some  applied  context.  Therefore,  each  enabling  objective  for  the  course 
Is  simultaneously  classified  as  remember  or  use,  aided  or  unaided,  and  transfer  or  not  transfer. 

Objective  and  Test  Adequacy  and  Consistency  Evaluation.  Next,  test  Items  from  the  course 
examination  are  matched  to  each  enabling  objective  using  a  matrix-type  format.  In  addition,  a 
wide  variety  of  additional  Information  Is  collected  and  associated  with  each  enabling  objective 
including  (a)  the  general  task  to  which  the  objective  applies;  (b)  whether  supporting  objectives 
are  required;  (c)  whether  the  objective  Is  "essential,"  "nlce-to  know,"  or  "unnecessary";  (d)  the 
training  goal  to  which  the  objective  applies;  (e)  whether  the  objective  includes  a  statement  of 
conditions,  standards,  and  actions;  (f)  whether  the  test  Item  matches  the  objective;  (g)  whether 
the  test  Item  matches  the  training  goal;  and  (h)  whether  the  test  Item  Is  adequate.  Preferably 
these  evaluations  are  to  be  carried  out  by  two  or  three  subject-matter  experts  in  the  content 
area  being  evaluated. 

Presentation  Evaluation.  This  component  of  the  CES  applies  to  the  evaluation  of  the  course 
instructor  delivering  Instruction  in  a  classroom  or  laboratory  setting.  Even  here,  however,  the 
evaluation  is  considerably  broader  than  Air  Force  or  Army  procedures  in  that  It  first  requires 
evaluating  the  Instructor's  performance  with  respect  to  its  consistency  with  the  objectives 
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Identified  In  the  previous  components  and  the  test  Items  on  the  final  course  examination.  Here, 
the  Instructor's  lesson  plans,  student  guides  and  classroom  presentation— generally  over  repeated 
lessons— are  studied  to  determine  If  the  Instruction  Including  media,  materials  and  handouts 
matches  the  objectives  and  test  Items.  Each  objective  Is  marked  complete.  Incomplete,  or  not 
applicable  on  each  of  the  following  dimensions:  whether  the  objective  was  clearly  stated, 
whether  practice  In  remembering  was  provided,  whether  feedback  was  given,  whether  examples  were 
provided,  whether  an  opportunity  to  practice  using  the  Information  was  afforded,  and  whether 
feedback  followed  this  practice  opportunity. 

Next,  the  presentation  of  each  objective  Is  rated  for  adequacy  by  Indicating  whether  the 
practice,  feedback,  and  examples  provided  for  each  objective  were: 

1.  separate  from  other  segments  of  the  Instruction 

2.  properly  Identified 

3.  clearly  stated 

4.  accompanied  with  student  help 

5.  job-oriented 

6.  sufficient  In  number 

7.  sufficient  to  detect  student  errors 

8.  provided  In  an  easy  to  hard  progression 

9.  provided  In  a  way  to  encourage  student  Involvement 

From  a  review  of  the  Instructor's  media,  materials  and  handouts,  and  classroom  observation, 
raters  respond  "Yes''  or  "No*  to  each  dimension  applicable  to  the  practice,  feedback  and  examples 
provided  for  a  specific  objective. 

The  third  and  final  part  of  the  Navy's  Instructor  evaluation  parallels  ATC  Form  281  and  its 
Army  equivalent.  This  phase  of  the  evaluation  employs  an  Instructional  Effectiveness  Checklist 
comprised  of  five  general  factors  and  47  Items.  These  general  factors  are: 

Learning  Orientation  {7  Items) 

Instructor  Behavior  (19  Items) 

Media  Materials  (5  Items) 

Envl ronment/Safety  (9  Items) 

Student  Behavior  (7  Items) 

Each  Item  Is  checked  either  "Yes"  or  "No."  No  space  Is  explicitly  provided  fwr  comments  on 
the  form,  although  the  margins  could  conceivably  be  used  for  this  purpose.  Unlike  the  Air  Force 
and  Army  forms,  this  form  de«otes  separate  factors  to  environment/safety  and  to  student 
behavior.  Additionally,  the  7  Items  devoted  to  student  behavior  (e.g..  Did  It  appear  the 
students  achieved  the  objective?  Old  It  appear  the  students  took  an  adequate  amount  of  notes? 
How  Involved  were  the  students  In  the  learning  process?)  have  no  counterparts  on  the  Air  Force 
and  Army  forms. 
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A  matching  of  Items  on  the  Instructional  Effectiveness  Checklist  with  the  teaching 
effectiveness  Indicators  Identified  In  Tables  1,  2,  3,  6,  and  7  reveals  that  50*  of  the  Items 
have  some  degree  of  correspondence  (as  compared  to  80S  for  the  Air  Force  form  and  64*  for  the 
Any  form),  after  items  pertaining  to  student  behavior  are  eliminated  (since  the  forms  of  the 
other  services  do  not  Include  this  dimension).  Also,  about  10  of  the  40  Items  included  In  this 
analysis  appear  to  be  military-specific  and,  therefore.  Inapplicable  to  the  research  In  public 
school  classrooms.  Finally,  It  can  be  noted  that,  contrary  to  Air  Force  and  Army  procedures,  the 
Navy's  Instructional  Effectiveness  Checklist,  as  well  as  the  other  components  of  the  Course 
Evaluation  System,  Is  recommended  and  one  presumes  largely  used  only  when  "serious  problems"  come 
to  the  foreground  to  justify  the  labor-intensive  process  Involved.  The  regularity  of  Its  use  and 
the  context  within  which  feedback  Is  provided  are  unclear  from  currently  available 
documentation.  The  Navy's  Instructional  Effectiveness  Checklist  appears  In  Appendix  0. 


V.  SURVEY  OF  OBSERVATION  AND  MEASURMENT  FORMATS  SUITABLE  FOR 
FEEDBACK  ENHANCEMENT 

This  section  surveys  several  measurement  formats  typically  used  In  recording  effective 
teaching  behaviors,  such  as  those  reviewed  in  Section  II,  with  particular  emphasis  on  the 

advantages  and  disadvantages  of  each  format  for  providing  Instructional  feedback.  This  section 
Includes  the  Identification  of  those  Individuals  who  might  serve  as  data  recorders/observers  In 
the  ATC  Instructional  environment,  along  with  a  prioritization  of  formats  using  the  criteria  of 
acceptability  to  users,  ease  of  Implementation,  cost,  objectiveness.  Increased  feedback, 

validity,  and  reliability.  The  measurement  formats  reviewed  and  compared  In  this  section  are: 

Time  Interval  Observation 
Likert  Scales 

Semantic  Differential  Scales 
Guttman  Scales 
Checklists 

As  the  technical  details  of  these  formats  are  amply  covered  In  textbooks  on  measurement  and 

testing  (see,  for  example,  Kublszyn  &  Borlch,  1987),  they  will  be  only  briefly  summarized  here 

for  the  purpose  of  comparing  the  unique  strengths  and  weaknesses  of  each  format  for  providing 
feedback  to  ATC  Instructors. 


Time  Interval  Observation 

One  measurement  tool  that  has  been  used  In  the  evaluation  of  teaching  Is  the  observation  of 
discrete  categories  of  teaching  behavior  sampled  over  Intervals  of  time.  Two  characteristics 
distinguish  various  observation  Instruments  of  this  type:  (a)  the  recording  procedure,  and  { b ) 
the  Item  content. 

Tools  for  observing  continuing  classroom  events  may  employ  either  of  two  recording 
procedures— sign  or  category.  A  sign  system  records  an  event  only  once  regardless  of  how  often 
It  occurs  within  a  specified  time  period.  The  behavior  Is  given  a  code  (e.g.,  "/"),  which  merely 
Indicates  Its  presence  or  absence  within  a  particular  block  of  time.  A  category  system,  on  the 
other  hand,  records  a  given  Instructor  behavior  each  time  it  appears  or  every  time  it  appears 
within  a  brief  designated  interval  of  time.-. and,-  hence,  ■*ovides  a  frequency  count  for  the 
occurrence  of  specific  behaviors,  rather  than  a  mere  indication  of  their  presence  or  absence.  A 
frequency  count  may  also  be  obtained  using  a  modified  sign  system,  called  a  rating  Instrument, 
which  estimates  the  degree  to  which  a  particular  behavior  occurs.  For  example.  Instead  of  simply 
noting  the  presence  or  absence  of  a  behavior,  a  rating  instrument  may  indicate  the  relative 
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frequency  at  which  the  behavior  occurs  on,  say,  a  l-to-5  scale,  with  *5"  indicating  a  high 
frequency  of  occurrence  (e.g.,  "occurs  often")  and  "1",  a  low  frequency  of  occurrence  (e.g., 
"never  occurs"). 

Tine  Interval  observation  systems  can  be  further  differentiated  on  the  basis  of  their  Item 
content.  Generally,  observation  Instruments,  whether  of  the  category,  sign,  or  rating  variety, 
focus  on  either  high-  or  low-inference  behaviors.  Those  Instruments  which  ask  an  observer  to 
judge,  for  example,  the  general  degree  of  a  teacher's  clarity,  variety,  or  task  orientation 
require  high  Inference,  since  the  Item  content  (e.g.,  the  word  "clarity")  does  not  specify 
discrete  behaviors  that  must  occur  In  order  for  a  teacher  to  be  considered  clear,  flexible,  or 
task-oriented.  In  this  Instance,  the  Item  content  is  Integrative  and  cumulative  In  nature, 
forcing  the  observer  to  make  judgments  about  the  behavior  being  observed  over  a  relatively  long 
period  of  observation.  On  the  other  hand,  observation  Instruments  that  name  specific  behaviors 
to  be  recorded,  such  as  "Instructor  asks  questions"  or  "Instructor  uses  examples,"  require  less 
Inference  on  the  observer's  part.  Low-Inference  Item  content  such  as  this  generally  reflects 
separate  and  distinct  units  of  behaviors  which  are  easy  to  observe  over  relatively  brief  spans  of 
time,  sometimes  consisting  of  only  a  few  seconds.  One  unique  feature  of  low-inference 
observation  systems  Is  that,  due  to  their  discrete  behavioral  content  and  brief  time  intervals, 
they  can  be  used  to  record  the  sequence  or  flow  of  classroom  behaviors  from  Instructor  to  student 
and  from  one  Interval  to  the  next.  It  should  be  noted,  however,  that  not  all  observation  systems 
are  either  high-  or  low-inference.  Some  combine  the  two  types  of  item  content,  while  others 
require  an  Intermediate  level  of  Inference  from  the  observer. 

Figures  1,  2  and  3  Illustrate  differences  In  the  recording  procedures  and  time  Intervals 
coononly  used  In  the  sign,  category  and  rating  types  of  observation  systems. 


Figure  1.  Sign  System.  (A  check  mark  is  given  to  the  behavior  that  best 
characterizes  the  Interval.) 
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Instructor 


Learner 


Category 

1 

2 

3 

4 

5 

6  7 

8  9 

10 

Total 

Reviews  or  summarizes 

0 

Uses  examples 

0 

Provides  advanced 
organizer 

1 

1 

2 

Asks  question 

2 

1 

12 

1 

16 

Demonstrates 

5 

22 

3 

30 

Gives  directives 

1 

5 

3 

4 

13 

Provides  feedback 
correctl ves 

0 

Responds 

1 

7 

■ 

14 

1 

31 

Initiates 

0 

Silence 

2 

1 

1 

2 

3 

9 

Total  0 

0 

2 

16 

30 

13  0 

31  0 

9 

101 

Figure  2.  Category  System  for  Recording  Sequential  Events. 
(A  tally  Is  made  each  time  the  behavior  occurs.) 
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1. 

Enthusiasm:  Enthusiastic  -  Apathetic 

Enthusiastic 

Neutral 

Apathetl c 

1  2 

3 

4 

5 

2. 

Classroom  Environment:  Supportive 

-  Nonsupportl ve 

Supportive 

Moderate 

Nonsupportlve 

1  2 

3 

4 

5 

3. 

Task  Orientation:  Focused  -  Unfocused 

Focused 

Moderate 

Unfocused 

1  2 

3 

4 

5 

4. 

Clarity:  Clear  -  Unclear 

Cl  ear 

Moderate 

Unclear 

1  2 

3 

4 

5 

5. 

Structuring:  Structural  -  Unstructural 

Structural 

Moderate 

Unstructural 

1  2 

3 

4 

5 

6. 

Variety  of  Methods:  High  -  Low 

High 

Moderate 

Low 

1  2 

3 

4 

5 

7. 

Cognitive  Variety:  Varied  -  Unvaried 

Varied 

Moderate 

Unvaried 

1  2 

3 

4 

5 

8. 

Amount  of  Criticism:  High  -  Low 

High 

Moderate 

Low 

1  2 

3 

4 

5 

Figure  3.  Rating  System  after  a  1-hour  Interval  of  observation. 

(The  appropriate  degree  of  behavior  Is  circled) 

An  Important  distinction  among  time  interval  observation  Instruments  Involves  differences  in 
length  of  the  observation  Interval.  An  observation  Interval  Is  the  period  of  elapsed  time 
allowed  for  a  single  discrete  observation  and  for  recording  the  results  of  that  observation. 
Recording  procedures  can  differ  widely  with  respect  to  length  of  Interval.  The  sign  method--or 
the  process  of  recording  the  presence  or  absence  of  a  behavior  within  an  Interval  of  time — Is  the 
most  conmon  among  observation  systems.  However,  this  simple  notion  can  take  on  different 
qualities  depending  on  the  length  of  the  Interval  of  observation  used.  Sign  systems  have  been 
constructed  with  relatively  long  time  Intervals  (from  about  15  minutes  to  an  entire  class  period) 
as  well  as  relatively  short  time  Intervals  (from  about  10  seconds  to  1  minute).  Generally, 
however,  when  a  less  dependent,  less  discrete  and  more  general  (higher-inference)  behavior  Is  the 
focus  of  Interest,  the  more  appropriate  Is  a  long  time  Interval.  Conversely,  the  more 
Independent,  more  discrete  and  more  specific  (lower-inference)  is  the  behavior  of  Interest,  the 
more  appropriate  Is  a  short  time  Interval.  This  reflects  the  notion  that  when  the  behavior  of 
Interest  Is  occurring  rapidly  and  Independently  of  other  behaviors,  a  short  interval  would  be 
needed  to  capture  frequent  shifts  In  Its  status.  When  the  behavior  of  Interest  occurs  relatively 
Infrequently  and/or  Its  recognition  Is  affected  by  other  events,  a  longer  Interval  would  be 
needed  before  Its  presence  or  absence  could  be  detected. 
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Another  measurement  issue  related  to  the  length  of  an  Interval  is  the  extent  to  which  the 
observer  processes  and  integrates  information  throughout  the  interval  or  simply  records  the 
behavior  present  at  the  exact  moment  of  observation.  Long  Interval  systems,  say  on  the  order  of 
10  minutes  or  longer,  generally  attempt  to  depict  the  behavior  that  best  characterizes  the  entire 
Interval.  On  the  other  hand,  short  Interval  systems,  say  on  the  order  of  1  minute  or  less,  tend 
to  force  observers,  whether  intentionally  or  not,  to  use  a  "shapshot"  technique  requiring  little 
or  no  processing  and  Integrating  of  Information  beyond  that  needed  to  recognize  a  behavior  at  an 
exact  moment  of  time.  These  two  types  of  Intervals  can  provide  different  results,  since  in  the 
former  case  the  accuracy  of  the  data  depends,  In  part,  on  the  abilities  of  the  observers  to 
process  and  to  Integrate  information  during  the  observation  Interval,  whereas  In  the  latter  case, 
the  accuracy  of  the  data  depends  on  the  ability  of  the  observers  to  simply  recognize  the  behavior 
at  a  given  Instant. 


Su— ated  Ratings 

Summated  ratings  can  be  divided  Into  Likert  and  Semantic  Differential  Scales. 

Likert  Scales.  The  Likert  scaling  technique  requires  a  large  number  of  Items  that  describe 
teaching  behaviors,  each  yielding  a  high  score  for  a  favorable  rating  on  a  behavior  and  a  lower 
score  for  a  less  favorable  rating.  The  rater  reacts  to  Items  on  a  5-polnt  response  continuum 
which  reflects  either  the  quality  of  behavior  or  the  frequency  at  which  It  was  perceived  to 
occur.  The  Likert  procedure  customarily  yields  scales  with  moderate  to  high  reliability. 
Validity,  however,  can  vary,  due  to  the  following  considerations: 

.  Generally,  no  attempt  Is  made  In  the  construction  of  a  Likert  scale  to  ensure  equal 
distances  between  units  (e.g.,  between  "very  often"  and  "fairly  often,"  or  between 
"always  relevant"  and  "most  relevant").  Therefore,  Increments  of  change  may  have 
different  meanings  at  different  parts  of  the  scale. 

.  The  unldlmenslonallty  of  the  scale  (l.e.,  the  extent  to  which  It  measures  a  single, 
distinct  behavior)  must  be  inferred  from  high  correlations  between  item  and  total 
scores.  Low  item/total  correlations  Indicate  that  the  construct  Is  too 
multifaceted  and  factorfally  complex  to  allow  simple  and  direct  Interpretation. 

.  Likert  scores  generally  are  Interpreted  according  to  a  distribution  of  sample 

scores,  and  an  Individual  Instructor's  score  has  meaning  only  in  relation  to  the 
scores  of  other  Instructors  who  have  been  similarly  rated.  In  other  words,  these 

scales  provide  ratings  that  are  relative  to  others  being  rated  and  not  to  some 

absolute  standard  or  criterion. 

A  typical  Likert  scale  Item  as: 

Instructor  uses  advanced  organizers 

_ very  often 

_ fairly  often 

_ sometimes 

_ fairly  rarely 

_ very  rarely 
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Semantic  Differential  Scales.  The  semantic  differential  scale  Is  another  sumuted  rating 
method  used  to  cumulatively  record  the  quality  or  frequency  of  teaching  behaviors.  It  requires 
the  rater  to  judge  the  Instructor's  performance  on  a  series  of  7-polnt  bipolar  scales.  The  rater 
checks  the  appropriate  space.  Indicating  both  the  direction  and  Intensity  of  the  judgment. 
Scores  are  derived  by  assigning  numbers  ( 1 . e. .  1,  2,  3,  4,  S,  6,  and  7)  to  each  position  on  the 
rating  scale.  Since  the  semantic  differential  and  Likert  scales  are  similar,  the  cautions  noted 
above  also  apply.  The  semantic  differential  type  does  not  necessarily  exhibit  equal  Intervals 
between  scale  points;  the  unidimensionality  of  the  concept  being  measured  may  vary  from  one  scale 
to  another  (particularly  when  bipolar  responses  are  not  exact  opposites);  and  scores  are 
Interpreted  relative  to  the  rated  performance  of  others.  In  practice,  differences  between  Likert 
and  semantic  differential  scales  are  minor  and  generally  are  related  to  the  use  of  5-  or  7-polnt 
response  formats.  The  similarity  of  these  procedures  Is  often  reflected  by  high  or  moderate 
correlations  between  the  two  when  they  are  used  to  measure  the  same  behavior.  A  portion  of  a 
typical  semantic  differential  scale  might  appear  as: 


Today's  Lesson 

Structured  _  _  _  _  _ 

Clear  _  _  _  _  _ 

Focused 


Unstructured 

Unclear 

Unfocused 


Scalogram  Analysis  (the  Suttman  Scale) 

Another  method  that  Is  used  to  record  judgment  of  teaching  performance  is  the  Guttaan  scale. 
This  method  Is  based  on  the  Idea  that  some  behaviors  can  be  arranged  hierarchically  so  that  an 
Instructor  who  manifests  a  particular  behavior  at  some  level  of  complexity  may  be  assumed  to 
possess  all  other,  related  behaviors  of  lesser  complexity.  For  example.  If  an  Instructor 
correctly  uses  visuals  In  the  course  of  teaching,  it  may  be  assumed  that  the  Instructor  also  can 
correctly  show  to  another  Instructor  how  visuals  can  be  used  in  a  written  lesson  plan  and  can 
recall  three  of  the  most  prominent  uses  of  visuals.  When  such  an  arrangement  Is  found  to  be 
valid,  the  behaviors  are  said  to  be  scalable. 

In  developing  the  Guttaan  scale.  Items  are  formulated  and  arranged  In  a  most  to  least 
complex— or  hierarchical —order.  These  Items  are  then  pilot-tested  on  a  group  of  Instructors, 
whose  response  patterns  are  then  analyzed  to  determine  whether  or  not  the  Items  are  scalable.  If 
Items  require  only  agreement  or  disagreement  (i.e.,  an  Indication  of  the  presence  or  absence  of  a 
behavior),  only  n  +  1  of  these  patterns  can  be  obtained.  The  relatively  low  frequency  of  deviant 
patterns  allows  the  co*>utat1on  of  what  Is  called  a  coefficient  of  reproducibility  (R).  R  is 
equal  to  the  proportion  of  responses  that  can  be  correctly  reproduced  from  the  knowledge  of  an 
Instructor's  score.  The  extent  to  which  such  Inferences  can  be  made  depends  upon  the  level  of 
the  coefficient  of  reproducibility.  This  value  represents  a  measure  of  the  unidimensional Ity  of 
the  scale  and  Is  an  Index  of  the  scale's  validity. 

Like  the  Likert  and  semantic  differential  scales,  the  Guttman  scale  makes  no  attempt  to 
ensure  equal  units  between  Items.  However,  unlike  the  Likert  and  semantic  differential  types, 
the  Guttman  scale  need  not  be  Interpreted  relative  to  the  ratings  of  other  Instructors,  since  its 
items  represent  specific  discrete  behaviors,  the  presence  or  absence  of  which  can  form  the  basis 
of  an  absolute  as  well  as  a  relative  judgment.  Generally,  this  Is  considered  a  desirable 
characteristic  for  measuring  teaching  competence.  A  typical  Guttman  Scale  appears  below. 
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0  Instructor  can  elicit  Increased  pupil  achievement  with  use  of  visuals. 

0  Instructor  can  correctly  use  visuals  In  the  course  of  teaching  lesson  content. 

0  Instructor  can  correctly  show  how  visuals  can  be  used  In  a  written  lesson  plan  or 

simulated  environment. 

0  Instructor  can  orally  explain  proper  use  of  visuals. 

0  • 

•  0 

«  « 

•  0 

Expected  Deviant 

pattern  pattern 

The  expected  pattern  Indicates  one  of  several  response  arrangements  wherein  the  Item  would  be 
considered  scalable  (high  coefficient  of  reproducibility,  If  found  to  occur  over  a  large  number 
of  Individuals).  The  deviant  pattern  Indicates  one  of  several  response  arrangements  wherein  the 
Item  would  be  considered  unscalable  (low  coefficient  of  reproducibility)  and  In  need  of  revision 
and  further  pilot-testing. 


Checklists 


When  a  behavior  cannot  be  easily  rated  on  a  continuum  of  values,  a  simple  Indication  of  Its 

presence  or  absence  Is  used.  If  an  observer  Is  unable  to  make  fine  gradations  In  judging  the 

quality  or  frequency  of  behavior,  a  simple  yes-no,  observed-unobserved,  or  present-absent  format 
Is  sometimes  used.  Since  checklists  record  only  the  presence  or  absence  of  behaviors,  they 
assume  that  the  rater  has  had  ample  opportunity  to  observe  the  behavior  In  question  and  the 
Instructor  has  had  ample  opportunity  and  time  to  display  the  behavior.  However,  this  assumption 

may  at  times  be  unwarranted.  When  the  checklist  data  Indicate  the  absence  of  a  particular 

behavior.  It  should  be  determined  whether  this  reflects  a  true  absence  or  simply  a  lack  of 

opportunity  to  observe  the  behavior.  The  latter  situation  may  occur,  for  example,  when  the 

Instructor's  objectives  are  unrelated  to  or  Incompatible  with  the  particular  behavior  in 

question,  or  when  the  rater  has  visited  the  classroom  too  Infrequently  or  for  too  short  a  period 

to  have  had  an  opportunity  to  observe  the  behavior.  In  order  for  the  rater  to  distinguish  the 
absence  of  an  event  from  Inadequate  opportunity  to  observe  the  event  checklists  should  provide 
two  separate  response  alternatives:  (a)  opportunity  to  observe  the  event,  and  (b)  presence  of 
the  event.  The  rater  would  check  the  first  alternative  whenever  a  behavior  on  the  checklist  can 
be  observed,  given  existing  classroom  conditions.  The  "true"  presence  or  absence  of  a  behavior 
would  then  be  recorded  by  checking  or  falling  to  check  the  second  alternative,  but  only  after  a 
check  has  Indicated  an  opportunity  to  actually  observe  this  behavior  exists,  given  the  existing 
classroom  conditions.  This  format  Is  Illustrated  below. 


Opportunity  Present 


0 


0 


Instructor  uses  advanced  organizers 


0 


0 


Instructor  reviews  and  sumnarlzes 


0 


0 


Instructor  provides  examples  and  demonstrations 
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The  following  summary  lists  the  advantages  and  disadvantages  of  each  of  the  above 
measurement/observation  formats. 


Time  Interval  Observation 
Advantages 

1.  Can  provide  a  large  representative  sampling  of  behavior. 

2.  Can  record  behavioral  sequence  of  Instructor-student  dialogue. 

3.  Can  provide  a  detailed  longitudinal  record,  week -by-week  or  month-by-month,  of  an 
Instructor's  performance. 

Disadvantages 

1.  Provides  large  amounts  of  data  that  must  be  reduced  through  statistical  analysis. 

2.  Requires  the  training  of  observers  to  assure  reliable  coding. 

3.  Can  be  time-consuming  to  process  and  Interpret  If  large  amounts  of  uata  are  collected. 


Rating  Scales 
Advantages 

1.  Directs  observation  toward  specific  and  clearly  defined  aspects  of  behavior. 

2.  Provides  a  coanon  frame  of  reference  for  comparing  all  Individuals  on  the  same  set  of 
characteristics. 

Disadvantages 

1.  Results  In  a  tendency  to  rate  all  Individuals  at  approximately  the  same  position  on  the 
scale. 


2.  Can  allow  a  rater's  general  Impression  of  the  person  to  Influence  how  that  person  Is 
rated  on  Individual  characteristics. 

3.  Usually  allows  for  relative  judgments  only;  that  Is,  allows  for  comparisons  among 
Individuals  but  not  absolute  judgments  that  a  particular  behavior  did  or  did  not  occur. 


Checklist 


Advantages 

1.  Is  useful  In  evaluating  those  performance  skills  that  can  be  divided  Into  a  series  of 
clearly  defined,  specific  actions. 

2.  Can  provide  absolute  judgments  of  the  presence  or  absence  of  specific  learning  outcomes. 
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01 sadvantages 


1.  Is  not  useful  In  recording  general  Impressions  of  behaviors  that  are  customarily  seen  as 
continuous;  for  example,  an  Instructor's  enthusiasm,  experience  or  attitude. 

2.  Can  provide  Inaccurate  data  when  the  opportunity  to  observe  the  behavior  Is  not  recorded 
at  the  time  of  scoring. 


Guttman  Scale 
Advantages 

1.  Due  to  its  hierarchical  construction,  can  provide  Information  as  to  whether  different 
behaviors  have  been  attained  without  actually  having  observed  all  of  them. 

2.  Provides  relative  as  well  as  absolute  ratings. 

01 sadvantages 

1.  Requires  considerable  knowledge  of  the  behaviors  being  measured  In  order  to  arrange  them 
Into  a  hierarchy. 

2.  Is  applicable  only  to  those  behaviors  that  can  be  related  to  one  another  In  a 
hierarchical,  stair-step  fashion. 

3.  May  require  extensive  pilot-testing 

Table  10  presents  these  measurement  systems  In  relation  to  five  different  groups  of 
Individuals  who  could  potentially  serve  as  observers  and/or  raters  in  an  enhanced  evaluation  and 
feedback  system.  These  groups  are:  the  designated  Instructional  supervisor  within  a  particular 
ATC  content  domain,  the  senior  Instructor,  peer  Instructors,  students  and  the  Instructor 
himself /herself  using  videotaped  episodes  of  his/her  own  teaching.  Theoretically  It  would  be 
possible  for  each  of  these  groups  or  combinations  of  them  to  provide  data  for  instructor  feedback 
using  one  or  more  of  the  measurement  formats  listed.  There  are,  however,  tradeoffs  that  would 
need  to  be  considered  In  selecting  any  measurement  system-data  provider  combination. 

Table  10.  Measurement  Systems  by  Possible  Data  Providers 

Supervisor  Senior  Peer  Videotape 

_ Instructor  Instructors  Students  Self  Evaluation 

Time 

Interval 

Observation 

Ratings 

Guttman 

Scale 

Check!  1st 
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Using  the  criteria  of  acceptability  to  the  user,  ease  of  Implementation,  cost,  objectivity, 
ami  Increased  feedback  and  validity  /reliability,  estimates  of  these  tradeoffs  can  be  made. 
First,  It  can  be  noted  that  greater  differences  in  tradeoffs  are  likely  to  occur  from  one 
measurement  system  to  another  (down  the  columns  of  Table  10)  than  from  one  data  provider  group  to 
another  (across  the  rows).  This  Is  to  say  that,  for  example,  acceptability  to  users  or  cost  Is 
likely  to  vary  considerably  more  across  measurement  systems  than,  say,  from  supervisors  to  senior 
Instructors.  Although  some  variation  among  these  groups  In  perceived  acceptability,  ease  of  use, 
cost,  objectivity,  and  Increased  feedback  and  validity /reliability  may  occur,  these  sources  may 
be  considered  moderately  or  highly  correlated  with  respect  to  the  judgment  criteria  being  used. 
Therefore,  the  following  description  sumnarlzes  the  most  salient  differences  across  measurement 
systems,  focusing  on  data  provider  groups  only  when  there  is  likely  to  be  a  significant  departure 
In  acceptability,  ease  of  use,  cost,  objectivity,  feedback,  and  validlty/rellabll Ity  among  these 
groups.  Although  somewhat  speculative,  the  following  section  points  out  some  of  the  advantages 
and  limitations  of  these  measurement  systems  for  enhanced  Instructor  feedback  in  the  ATC 
1 nstructl onal  envl ronment . 


Some  Advantages  and  Limitations  of  Time  Interval  Observation 

Time  interval  observation  often  Is  considered  among  the  most  expensive  methods  of  providing 
observation  and  feedback  data.  For  this  reason,  Its  use  has  been  limited  largely  to  classroom 
research  on  teaching  effectiveness.  Its  cost  derives  largely  from  the  extensive  training  and 
coding  manuals  that  are  often  necessary  to  secure  high  levels  of  reliability  by  those  doing  the 
coding.  This  also  reduces  Its  ease  of  use  In  comparison  of  other  measurement  systems  Inasmuch  as 
training  workshops  and  material  (manual)  development  is  usually  mandatory  before  coders  can 
become  proficient  at  perceiving  and  categorizing  classroom  behavior  at  a  relatively  rapid  rate 
(e.g.,  10-,  20-  or  30-second  intervals).  This  problem  becomes  less  severe  when  a  sign  system  (as 
opposed  to  a  category  system)  Is  used  and  time  Intervals  may  be  extended  to  10  or  15  minutes. 
However.  In  this  case,  the  system  functions  much  like  a  simple  checklist.  When  sufficient 
resources  have  been  devoted  to  coder  training  and  manual  development,  the  objectivity,  amount  of 
feedback  and  reliability  of  a  time  Interval  system  is  generally  considered  to  be  high.  However, 
little  Is  known  about  the  validity  of  these  systems  except  that  the  data  provided  by  them 
sometimes  vary  depending  on  the  length  of  the  time  sampling  Interval  chosen,  which  is  often 
arbitrary.  Finally,  It  Is  Important  to  note  that  time  Interval  observation,  especially  when  the 
length  of  the  Interval  Is  small  can  produce  voluminous  frequency  counts  of  Individual  behaviors, 
and  can  easily  befuddle  and  overwhelm  a  naive  user.  This  Is  yet  another  reason  why  time  Interval 
observation  has  found  greater  acceptance  among  classroom  researchers  than  practitioners  who  may 
be  expected  to  use  or  feed  back  the  results  of  the  measurement  system  In  a  real-time 
environment.  There  Is  likely  to  be  little  difference  among  these  evaluative  criteria  across  data 
provider  groups,  except  to  note  that  this  measurement  system  would  not  be  suitable  for  use  by 
students  due  to  Its  time  and  technical  demands.  It  would,  on  the  other  hand,  be  more  suitable 
for  use  by  the  Instructors  themselves  using  videotaped  sej^nts  of  their  own  or  their  peers’ 
teaching  behaviors.  In  summary,  a  time  Interval  observation  system  can  provide  extensive  amounts 
of  specific,  discrete  data  about  an  Instructor's  use  of  specific  behaviors  via  frequency  counts. 
On  the  other  hand,  a  time  Interval  observation  system  Is  a  form  of  measuring  classroom  behavior 
that  may  be  cumbersome  to  use  and  that  is  likely  to  place  considerable  technical  and  time  demands 
on  Its  uers.  Its  use  seems  primarily  suited  for  classroom  research  and  only  secondarily  for 
Instructional  feedback. 


Suwated  Ratings  (Likert  and  Semantic  Differential) 

Likert  and  semantic  differential  scale- type  ratings  are  among  the  most  popular  forms  of 
instructor  assessment.  The  quality  of  feedback  Instructors  typically  receive  from  these  types  of 
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ratings,  however,  has  been  known  to  vary  depending  on  the  thought,  effort,  and  knowledge  of  the 
Instructional  process  that  has  guided  development  of  the  form  Itself.  Generally,  acceptability 
among  users  Is  moderate  to  high  with  these  types  of  systems,  since  most  Instructors  have  long 
been  accustomed  to  using  them  and  being  evaluated  by  them.  Their  acceptability,  however,  tends 
to  be  higher  for  evaluation  purposes  than  for  feedback,  in  that  most  summated  types  of  ratings 
tend  to  provide  only  relative  and  not  absolute  assessments  of  a  particular  teaching  behavior. 
Thav  Is,  they  most  typically  Indicate  that  more  or  less  of  a  given  behavior  Is  being  observed 

relative  to  others  who  are  being  rated,  but  not  whether  the  behavior  Is  at  some  predesignated 
standard  of  acceptable  performance.  This  means  that  moderate  or  even  sometimes  high  ratings  can 
mask  poor  performance  when  the  behavior  observed  for  an  Individual  Is  actually  only  slightly 
better  than  the  typical  but  poor  behavior  of  others  who  are  rated  with  the  same  scale.  This 

tends  to  make  simulated  ratings  more  useful  for  making  comparisons  across  Individuals  than  for 
providing  explicit  feedback  to  a  single  individual  as  to  which  effective  teaching  behaviors  were 
or  were  not  present.  All  simulated  ratings  need  not  be  only  relative  assessments,  however. 
Constructing  scale  Items  that  Identify  explicit  teaching  behaviors  derived  from  the  research 
literature  on  effective  teaching  and  reducing  scale  Intervals  to  only  those  variations  of  the 

behavior  that  can  be  reliably  detected  are  ways  in  which  summatlve  ratings  have  been  made  more 
useful  for  providing  Instructional  feedback. 

Finally,  It  can  be  noted  that  since  sumnated  ratings  often  take  the  form  of  relative 

assessments,  they  are  known  to  be  Influenced  by  who  Is  doing  the  rating  and  who  Is  being  rated. 
That  Is,  the  rater's  general  Impression  of  the  person  may  Influence  how  the  subject  Is  rated  and, 
conversely,  the  characteristics  of  the  rater  and  his/her  belief  or  value  system  can,  to  lesser  or 
greater  degrees.  Influence  the  ratings.  This  problem  Is  compounded  by  the  tendency  to  rate  most 
or  all  Individuals  at  approximately  the  same  position  on  the  scale,  due  to  the  rater's  belief  In 
how  the  data  may  be  used  to  help  or  hinder  particular  Individuals.  Hence,  the  lack  of  variation 
In  sumnated  ratings  can  prevent  the  provision  of  meaningful  feedback  to  the  person  being  rated. 

In  summary,  sumnated  ratings  (l.e.,  Likert  and  semantic  differential  scales)  are  easy  to  use, 
are  relatively  Inexpensive  to  construct  and  are  a  familiar  and  readily  acceptable  form  of 
measurement  to  most  users.  Summated  ratings  are  vague  or  abstract  allowing  the  opportunity  for 
personal  beliefs  and  values  to  Influence  the  rating.  Aiso,  since  most  summated  ratings  are 
assessments  of  the  degree  to  which  a  particular  behavior  occurs  relative  to  others  who  are  being 
rated,  as  opposed  to  identifying  the  absolute  presence  or  absence  of  a  particular  teaching 
practice,  they  tend  to  be  more  useful  for  evaluation  than  for  feedback.  It  can  be  noted, 
however,  that  with  proper  pilot-testing,  summated  ratings  frequently  produce  moderate  to  high 
reliabilities  and  modest  predictive  validity.  Differences  among  these  criteria  are  not  likely  to 
occur  across  the  data  provider  groups  Identified  in  Table  10. 


Some  Advantages  and  Limitations  of  the  Checklist 

Checklists  typically  have  been  the  easiest  to  use  and  most  acceptable  to  users.  Their  ease 
of  use  derives  largely  from  their  binary  (present,  absent)  nature  and  the  Increased  certitude 
raters  generally  experience  In  having  to  make  distinctions  of  only  a  “yes,  no,"  “present, 
absent,"  "observed,  unobserved"  variety,  when  Item  content  Is  sufficiently  discrete  as  to  lend 
itself  to  such  all-or-none  judgments.  This  has  created  both  respect  for  and  criticism  of  the 
checklist  technique  among  various  user  groups,  depending  on  the  nature  of  the  item  content  and 
the  purpose  for  which  the  data  are  to  be  used.  Generally,  when  Item  content  is  more  readily 
perceived  as  a  continuously  occurring  behavior  in  real  life  and  when  data  are  to  be  used  for 
evaluation  purposes,  the  checklist  format  has  not  been  the  method  of  choice.  Individuals  rarely 
prefer  to  rate  or  see  themselves  rated  on  an  all-or-none  basis  when  natural  variation  normally 
occurs  and  can  be  pointed  to  in  a  real-time  environment.  In  these  cases,  the  checklist  suffers 
from  overly  reducing  the  complexity  of  performing  a  teaching  act  by  making  trivial  or  simplistic 
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distinctions  that  do  not  normally  occur  In  the  classroom.  On  the  other  hand,  when  certain 
behaviors  do  occur  as  discrete  events,  and  degrees  of  variation  are  difficult  to  detect  reliably, 
the  measurement  of  that  behavior  Is  more  suited  to  a  checklist.  Furthermore,  when  decision 
making  Is  reduced  to  the  presence  or  absence  of  a  behavior,  feedback  becomes  absolute,  as  opposed 
to  relative,  enhancing  the  diagnostic  value  of  the  checklist  data.  That  Is,  recomnendatlons 
become  explicit  and  Instead  of  Informing  the  Instructor  that  relative  to  the  others  he/she  Is 
higher  or  lower  on  that  behavior,  the  instructor  Is  told  that  a  specific  competency  has  or  has 
not  been  observed. 

For  these  reasons.  If  used  with  the  proper  behavioral  content  and  for  feedback  or  diagnostic 
purposes,  the  checklist  Is  generally  acceptable,  easy  to  use,  relatively  cost  efficient,  and 
capable  of  providing  useful  feedback.  On  the  other  hand.  Its  objectivity  and  validity  depend 
heavily  on  the  relationship  between  the  behavior  being  observed  and  some  externally  anchored 
measure  of  teaching  effectiveness,  such  as  student  achievement. 

The  research  evidence  does  provide  some  links  between  the  presence  and  absence  of  individual 
teaching  behaviors  (e.g.,  use  of  advance  organizers,  questioning,  review  and  summarization)  and 
sti'dent  achievement,  as  indicated  In  the  review  of  the  literature  in  Section  II  of  this  paper, 
some  of  which  tends  to  support  a  checklist  format.  For  these  types  of  behaviors  It  would  be 
unlikely  or  difficult  to  observe  extensive  variation,  or  such  variation  would  not  be  deemed 
useful  for  diagnosis  and  feedback.  It  can  be  presumed,  however,  that  not  every  effective 
teaching  behavior  would  conform  to  this  characteristic.  Finally,  it  can  be  noted  that  due  to  the 
restricted  response  range  for  checklists,  reliabilities  can  be  among  the  highest  among  the 
measurement  systems  being  reviewed  here,  when  the  content  being  measured  Is  discrete.  In  at 
least  some  Instances,  simulated  ratings  have  shown  less  than  satisfactory  reliability  because  Item 
scales  were  asking  for  more  discrimination,  however  theoretically  justified,  than  observers  could 
practically  detect  In  a  real-time  environment.  With  respect  to  data  provider  groups.  It  would 
appear  that  the  checklist  would  be  equally  applicable  to  all  groups,  with  the  exception  that 
checklists  are  sufficiently  simple  to  use  In  real  time  that  they  would  not  require  the  use  of 
videotapes  nor  would  their  reliability  likely  be  enhanced  by  them. 


Some  Advantages  and  Limitations  of  the  Guttman  Scale 


The  Guttman  scale  is  considered  among  the  most  psychometrlcally  sophisticated  teacher 
evaluation  tools.  Its  sophistication,  however,  comes  at  considerable  cost  in  both  the  expertise 
needed  to  devise  hierarchical  scales  and  the  pilot-testing  needed  to  obtain  sufficient  degrees  of 
scalability.  Perhaps  foremost  among  Its  strengths,  however.  Is  Its  capacity  to  provide  data  for 
the  diagnosis  of  teacher  competencies,  assuming  the  competencies  are  scalable.  Rather  than 
providing  relative  ratings,  as  do  many  measurement  tools,  the  Guttman  scale  requires  the 
Identification  of  teaching  practices  for  which  discrete  classroom  operations  can  be  observed 
along  some  type  of  contlnua  (e.g.,  simple  to  complex).  The  Idea  behind  the  Guttman  scale  Is  that 
ultimate  competency  In  an  Individual  behavior  can  be  obsarved  In  a  stair-step  or  building-block 
fashion,  with  each  earlier  step  representing  a  necessary  requisite  part  of  each  subsequent  step. 
In  this  manner,  knowledge  as  to  where  an  Individual  Instructor  falls  on  any  step  automatically 
reveals  what  other  steps  have  and  have  not  been  achieved.  Hence,  with  the  observation  of  a 
single  operation.  Information  about  all  other  operations  pertaining  to  that  behavior  is 
simultaneously  obtained. 

As  was  noted,  wh^le  the  amount  of  feedback  and  diagnostic  Information  is  high  with  a  Guttman 
scale,  other  practical  considerations  often  temper  its  use.  Some  training  of  observers  is 
generally  considered  Important  for  obtaining  reliable  scales,  as  Is  some  undergirding  theory  or 
rationale  that  can  reveal  the  individual  steps  or  blocks  that  might  go  together  to  comprise  an 
item  seal*.  Without  a  keen  understanding  of  the  teaching  practice  being  assessed,  pilot-testing 
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Is  reduced  to  an  unsystematic  process  of  trial  and  error  which  attempts  to  find  the  scalable 
components  of  a  behavior.  Also,  It  Is  safe  to  presume  that  not  all  effective  teaching  behaviors 
will  have  scalable  components,  In  which  case  the  method  of  choice  would  be  a  checklist  or 
summated  rating.  In  addition,  acceptability  and  ease  of  use  of  the  Guttman  scale  may  be 
complicated  by  the  fact  that  many  practitioners  are  unfamiliar  with  Its  use.  This  may  affect  Its 
objectivity  and  reliability  unless  some  type  of  orientation  and/or  training  Is  given  to  users. 
As  with  all  measurement  systems,  the  validity  of  the  Guttman  scale  rests  on  Its  capacity  to 
predict  some  subsequent  Index  of  teaching  effectiveness.  Generally  speaking,  properly  scaled 
Guttman  items  have  produced  relationships  to  student  achievement  that  have  been  among  the  highest 
of  any  measurement  system.  Due  to  the  technical  demands  and  content  expertise  required  of  the 
Guttman  scale,  however.  Its  use  has  been  limited  to  well -trained  evaluators. 

The  following  table  ranks  the  four  measurement  systems  according  to  the  criteria  of 
acceptability  to  users,  ease  of  use,  cost,  objectivity,  feedback,  and  val Idlty/rel labil Ity. 

Table  11.  Ranking  of  Each  of  Four  Measurement  System  Against 
the  Six  Criteria 


Rank  Accept¬ 
ability 

Ease 

Cost 

Objectivity 

Feedback 

Validity/ 

Reliability 

1.  Ratings 

Checklist 

Checklist 

Time 

Tim 

Checklist 

Interval 

Interval 

Observation 

Observation 

2.  Checklist 

Ratings 

Ratings 

Checklist 

Guttman 

Ratings 

3.  Guttman 

Guttman 

Time 

Guttman 

Checklist 

Tim 

Interval 

Interval 

Observation 

Observation 

4.  Time 

Time 

Guttman 

Ratings 

Ratings 

Guttman 

Interval 

Interval 

Observation 

Observation 

From  the  above  table.  It  can  be  noted  that  the  number  of  criteria  for  which  a  particular 
measurement  system  appears  In  the  top  2  ranks  Is  as  follows: 

N;  ,nber  of  criteria  for  which 

System _ system  Is  Ranked  1  or  2 

Checklist  5 

Ratings  4 

Time  Interval  2 

Observation 

Guttman  1 

Overall,  checklists  and/or  summated  ratings  tend  to  have  the  most  potential  for  feedback 
enhancement.  Given  that  each  of  these  tools  has  Its  own  measurement  limitations  and  should  not  be 
judged  more  or  less  appropriate  apart  from  the  particular  teaching  behavior  it  purports  to 
measure,  some  combination  of  summated  ratings  and  checklist  would  seem  more  prudent  than  the 
exclusive  use  of  a  single  measurement  format. 
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VI.  SUMARY  AM)  RECOMIENDATIONS 


This  section  summarizes  the  results  of  this  Investigation  and  provides  recommendations  for 
revising  and/or  developing  procedures  for  enhancing  ATC  Instructor  feedback. 

*25 *2. 

Specifically,  the  goals  of  this  project  were: 

1.  To  survey  the  state-of-the-art  Indicators  of  teaching  effectiveness  reported  In  the 
research  literature. 

2.  To  describe  the  Implementation  of  teaching  effectiveness  Indicators  and  related  procedures 
being  used  to  evaluate  teachers  In  the  public  schools. 

3.  To  provide  an  overview  of  the  ATC  Instructor  evaluation  procedures  and  similar  procedures 
In  use  by  the  Army  and  Navy. 

4.  To  survey  and  prioritize  observation  and  measurement  formats  suitable  for  feedback 
enhancement  In  the  ATC  Instructional  environment. 

The  purpose  of  Investigating  these  four  areas  was  to  present  a  comprehensive  set  of  data  by 
which  the  Air  Force  could  determine  the  feasibility  and  desirability  of  revising  and/or  extending 
existing  ATC  Instructor  evaluation  procedures  to  enhance  feedback  to  the  Instructor.  The  data, 
their  Implications,  and  the  recommendations  resulting  from  this  report  together  represent  a  basis 
for  deciding  whether  additional  research  and  development  work  would  benefit  this  feedback 
enhancement  goal . 

Phase  1  of  this  project  surveyed  the  state-of-the-art  Indicators  of  teaching  effectiveness 
reported  In  the  research  literature.  In  this  phase  a  comprehensive  review  of  the  results  of 
classroom  research  on  effective  teaching  from  1960  to  the  present  was  undertaken.  Five  general 
dimensions  of  teaching  effectiveness  consisting  of  28  distinct  teaching  behaviors  were  identified 
as  having  been  consistently  associated  with  Increases  In  student  achievement  on  standardized  and 
classroom  tests  of  academic  performance.  These  28  behaviors,  along  with  rationale  for  their 
effectiveness,  were  provided  In  Section  II  of  this  paper. 

Phase  2  of  the  Investigation,  then,  turned  to  the  extent  to  which  the  public  schools  were 
actually  using  these  and/or  other  Indicators  of  teaching  effectiveness  to  evaluate  and  provide 
feedback  to  classroom  Instructors.  It  was  found  that  the  majority  of  states  require  a  performance 
appraisal  and  feedback  system  of  Individual  school  districts  using  at  least  some  research-based 
teaching  effectiveness  Indicators  similar  to  or  Identical  with  those  reviewed  In  Section  II.  In 
one  example,  utilizing  the  performance  appraisal  system  required  by  the  State  of  Texas,  79*  of  the 
28  effective  teaching  behaviors  Identified  from  the  research  literature  had  counterparts  on  that 
State's  appraisal  Instrument.  Typically,  state  appraisal  instruments  often  Include  other, 
professionally  based  teaching  behaviors  In  addition  to  those  found  in  the  research  literature. 

Phase  3  of  this  investigation  examined  the  current  ATC  evaluation  procedures  for  their 
feedback  potential,  with  specific  reference  to  Form  281  and  Form  736,  the  student  critique.  It 
then  compared  these  procedures  with  those  currently  In  use  by  the  Army  and  Navy.  For  this  task 
the  contractor  observed  classroom,  laboratory  and  field  instruction  across  18  different 
specialties  at  four  technical  training  centers  (Lackland,  Keesler,  Sheppard,  and  Lowry  AFBs). 
After  each  observation,  the  Instructor  and  one  or  more  instructor  supervisors  were  interviewed  as 
to  their  opinions  about  the  capacity  of  Forms  281  and  736  to  provide  useful  feedback  and  other 
topics  related  to  feedback  enhancement.  In  a  separate  analysis.  It  was  determined  that  80*  of  the 
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teaching  behaviors  on  ATC  Form  281  were  found  to  overlap  those  found  In  the  research  literature 
reviewed  In  Section  II.  In  comparison,  64S  of  the  behaviors  on  the  Army  Instructor  appraisal  form 
and  501  of  the  behaviors  on  the  Navy  appraisal  form  had  counterparts  In  the  research  literature. 
The  Air  Force  procedures  for  evaluating  Instructor  effectiveness  appeared  sufficiently  Inclusive 
as  to  Incorporate  the  most  significant  elements  of  evaluation  In  use  by  the  other  services. 
Additionally,  this  phase  of  the  Investigation  found  that  Instructors  and  supervisors  believed  that: 

1.  Several  of  the  Items  on  Form  281  were  either  redundant  or  highly  correlated. 

2.  Some  Items  on  Form  281  were  not  under  the  control  of  the  Instructor. 

3.  Some  Items  would  be  more  appropriately  scaled  with  a  2-polnt  format  (checklist)  than  a 
4-point  format. 

4.  Some  areas  Important  to  an  Instructor's  effectiveness  (e.g.,  pertaining  to  the  cognitive 
organization  of  the  Instruction  and  providing  Impromptu  examples)  were  absent  and  some  (e.g., 
pertaining  to  public  speaking  and  communication)  were  overemphasized  on  Form  281. 

5.  In  the  absence  of  more  explicit  definitions,  some  Items  on  Form  281  were  ambiguous  and 
subjective. 

6.  Form  281  Is  significantly  less  suitable  and  efficient  for  evaluation  and  feedback  In  a 
laboratory  or  field  class  than  In  a  formal  stand-up  classroom  lecture. 

Phase  4  of  the  Investigation  surveyed  the  advantages  and  disadvantages  of  several  types  of 
measurement  formats  that  might  be  used  to  provide  additional  feedback  to  ATC  Instructors.  The 
measurement  formats  reviewed  were: 

1.  Time  Interval  sampling 

2.  Likert  scales 

3.  Semantic  differential  scales 

4.  Checklists 

5.  Guttman  scales 

Each  format  was  reviewed  for  Its  acceptability  to  users,  ease  of  implementation,  cost, 
objectivity,  and  Increases  to  feedback,  validity  and  reliability  (see  Table  11,  p.  63).  In 
addition,  a  list  of  possible  ATC  data  providers  consisting  of  supervisor,  senior  instructor,  peer 
Instructors,  students,  and  videotape  self  evaluation  were  discussed  In  relation  to  the 
practicality  of  these  Individuals  participating  in  the  feedback  enhancement  process  using  these 
various  measurement  formats.  In  addition  to  prioritized  rankings  of  each  measurement  format  under 
each  of  the  above  six  criteria  (acceptabll Ity,  ease,  cost,  objectivity,  feedback,  validity  and 
reliability),  the  results  Indicated  that  checklists  and  Likert  scales  ./ould  be  the  most  suitable 
and  practical  for  a  feedback  enhancement  system. 


Recommendations 


Recommendations  pertain  to  the  following  six  areas  of  ATC  Instructor  evaluation  and  feedback: 
1.  ATC  Form  281 
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2.  Supervisor's  training  course 

3.  Evaluation  of  laboratory  and  field-based  Instruction 

4.  Peer  evaluation  of  Instructors 

5.  Student  evaluation  of  Instructors 

6.  Technical  training  Instructor  course 

Specific  development  Initiatives  Mill  be  suggested  In  each  of  these  areas,  along  with  an 
approximate  estimate  of  the  number  of  man-days  required  to  complete  each  development  task, 
exclusive  of  pilot-testing  and  validation. 


ATC  Fora  281 


ATC  Form  281  Is  the  major  source  of  Instructor  feedback  currently  used  by  the  Air  Force. 
Although  most  Instructors  and  Instructor  supervisors  agree  that  Form  281  appears  adequate  In  Its 
present  fora  for  evaluation  purposes,  the  consensus  Is  less  strong  that  Form  281  comprises  an 
adequate  Instrument  for  feedback  to  the  Instructor  as  to  his  or  her  teaching  effectiveness. 
Problems  pertaining  to  the  feedback  purpose  of  Form  281  which  were  mentioned  during  Interviews 
with  Instructors  and  Instructor  supervisors  were: 

1.  Some  Items  on  Fora  281  may  not  a  ways  be  under  the  control  of  the  Instructor.  Some  Form 
281  Items  frequently  mentioned  were:  A2  "Classroom  neat  and  orderly,  seating  arrangement 
appropriate.  Items  Identified  needing  repair,  such  as  burned  out  lights";  and  II  “Students  In 
compliance  with  appropriate  dress  and  appearance  standards."  Instructors  who  rotate  from 
classroom  to  classroom  due  to  space  and  scheduling  limitations  (about  50*  of  the  ATC  Instructors 
sampled)  are  quick  to  point  out  the  Irrelevance  of  Item  A2  for  feedback  (and  evaluation). 

2.  Some  ATC  Form  281  Items  appear  redundant  or  highly  correlated.  Some  Instructor 
supervisors,  for  example,  reported  having  difficulty  distinguishing  among  such  Items  as  “Student 
participation  encouraged“  (H5)  and  “Group  members  encouraged  to  participate*  (12).  Although  the 
latter  Involves  groups,  the  latter  subsumes  the  former,  thereby  providing  a  lack  of  distinctive 
Information  across  these  Items.  In  addition,  some  supervisors  report  that  Item  G3  ("Variety  of 
question  types  used.  Types  of  questions  used  adjusted  to  situation.")  is  another  form  of 
encouraging  student  participation  and,  therefore,  highly  related  to  Items  H5  and  12,  sometimes 
making  meaningful  distinctions  among  these  Items  difficult. 

3.  Some  ATC  Form  281  Items  appear  more  suited  to  a  present/absent  scale  than  to  the  currently 
used  4-polnt  rating  system.  For  example,  some  instructor  supervisors  see  no  distinction  between  a 
rating  of  "Satisfactory1’  and  one  of  "Outstanding"  for  Items  which  primarily  are  intended  to  record 
whether  a  standard  or  regulation  has  been  met  or  not  met.  Cited  by  Instructor  supervisors  were 
Items  such  as  A1  ("Lesson  plan  current,  personalized,  and  appropriate.  Lesson  plan  signed  by 
Instructor's  supervisor." )  and  A3  ("Training  materials,  aids,  and  equipment  available  prior  to  the 
start  of  the  class.")  Instructor  supervisors  frequently  reported  answering  these  types  of 
questions  as  “Yes"  or  "No,"  thereby  arbitrarily  rating  the  Instructor  either  as  "Outstanding"  or 
“Satisfactory,"  when  a  “Yes"  response  was  in  order.  Some  supervisors  give  only  a  "Satisfactory" 
rating  when  performance  meets  the  implied  standard,  while  other  supervisors  customarily  give  an 
“(Xitstandlng"  rating  when  the  condition  has  been  met,  leading  to  Inconsistencies  across  supervisor 
ratings  for  these  types  of  Items. 
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4.  Some  Instructors  and  supervisors  reported  that  ATC  Form  281  tends  to  e^haslze  soae  of  the 
most  easily  observable  characteristics  of  presenting  (e.g.,  eye  contact,  gestures,  enthusiasm)  and 
communicating  (e.g.,  enunciation,  mannerisms,  voice  quality)  at  the  expense  of  other  qualities 
which  may  have  as  much  In  conaon  with  effective  teaching  as  good  speaking.  Although  every 
Instructor  and  supervisor  Interviewed  reported  on  the  close  and  necessary  relationship  between 
these  two  areas,  some  considered  the  absence  or  underemphasls  of  Items  such  as  the  ones  pertaining 
to  (a)  the  accuracy  of  the  content  being  presented,  (b)  the  timeliness  or  recency  of  examples  used 
to  Illustrate  major  points,  (c)  the  ability  of  the  Instructor  to  use  unexpected  opportunities 
(such  as  particularly  adept  and  unusual  student  response)  to  spontaneously  foster  lesson 
objectives,  and  (d)  the  use  of  advanced  organizers  to  cognitively  "frame"  the  lesson,  as 
limitations  of  the  usefulness  of  Form  281  for  Instructor  feedback. 

5.  It  was  noted  by  both  Instructors  and  supervisors  that  few  guidelines  exist  or  have  been 

conmunlcated  as  to  the  extent  and  nature  of  the  coaments  that  should  be  provided  on  Form  281. 
Although  any  Item  receiving  a  rating  of  “Outstanding"  or  "Meeds  Improvement"  must  receive  a 

comment  In  the  right-hand  margin,  no  specific  guidelines  or  exanples  exist  to  encourage  the 
feedback  purpose  on  this  portion  of  Form  281.  As  a  result,  feedback  In  the  form  of  comments  on 
Form  281  varies  considerably  across  Instructors,  courses  and  technical  training  centers.  A 
perusal  of  a  sample  of  Forms  281  Indicated  that  some  supervisors  comment  on  Items  rated 
"Outstanding"  and  "Needs  Improvement"  as  specified  In  the  regulations  but  seldom  or  never  on  Items 
rated  "Satisfactory."  Other  supervisors  seem  to  prefer  to  comment  extensively  on  many  Items  In 
some  areas  of  Form  281  but  not  at  all  on  Items  In  other  areas.  Generally,  the  Instructors  find 
the  written  comments  helpful,  but  sometimes  Inadequate  explanation  as  to  the  precise  reason  why  a 

behavior  was  rated  the  way  that  It  was.  Often  these  comments  are  simply  congratulatory  If  a 

behavior  was  well  executed  (but  do  not  say  what  made  It  exemplary)  or  simply  a  call  for 

Improvement  when  the  behavior  was  inadequate  (but  do  not  Indicate  how  to  improve  It). 

The  above  observations  lead  to  the  following  suggestions  for  revising  ATC  Form  281  to  enhance 
Its  Instructional  feedback  characteristics: 

1.  Reword  Items  on  Form  281  to  reflect  only  those  responsibilities  that  are  typically  under 
the  control  of  the  Instructor.  Delete  references  to  student  behavior  that  may  be  only  Indirectly 
or  tangentially  related  to  effective  classroom  Instruction  and,  therefore,  not  typically  under  the 
control  of  the  Instructor. 

2.  Partition  (e.g.,  adherence  to  standards  and  regulations)  Form  281  items  Into  those  most 
suitable  to  a  checklist  and  those  for  which  reliable  discriminations  can  be  made  on  the  present 
4-polnt  scale.  Reorganize  Form  281  accordingly. 

3.  Replace  r  ndant  Items  on  Form  281  with  additional  Items  that  reflect  the  technical 
accuracy  (e.g.,  K1  "Thorough  knowledge  of  subject  matter  demonstrated"  might  be  replaced  with 
distinctions  such  as  knowledge  of  mechanics,  of  concepts,  of  applications,  of  rules  and 
regulations,  etc.),  timeliness,  spontaneity  and  cognitive  organization  of  the  Instructor's 
lesson.  Add  any  behaviors  from  the  effective  teaching  literature  that  are  consistent  with  the  12 
general  dimensions  which  currently  appear  on  Form  281. 

4.  Provide  written  guidelines  for  and  realistic  examples  of  comments  reflecting  diagnostic 
feedback  and  suggested  sources  of  remediation  for  each  Item  on  Form  281.  Make  this  written 
documentation  available  through  the  faculty  development  office  of  each  Wing  and  the  supervisor's 
training  course. 

The  above  revisions  could  be  first  pilot-tested  as  a  new  peer  evaluation  form,  to  be  used  on  a 
voluntary  basis  over  the  course  of  12  months,  the  data  from  which  could  then  be  used  for 
evaluating  any  increase  in  feedback  yielded  by  these  changes.  It  is  estimated  that  approximately 
60  man-days  would  be  required  for  completing  these  changes,  exclusive  of  pilot-testing. 
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Supervisor  Training  Course 

The  supervisor  training  course  Is  designed  to  acquaint  master  Instructors  with  the  supervisory 
and  adalnl strati ve  duties  they  are  about  to  assume  as  Instructor  supervisors.  Although  the 
composition  of  the  course  and  areas  of  emphasis  within  It  tend  to  vary  slightly  to  reflect  the 
needs  specific  to  the  technical  training  center  at  which  It  Is  taught,  currlculua  for  the  course 
specifies  that  an  Introduction  to  the  use  of  ATC  Form  281  must  be  taught.  Generally,  this 
Introduction  consists  of  a  10-minute  film  Illustrating  an  Air  Force  Instructor  presenting  a  formal 
classroom  lecture  In  which  several  Ineffective  teaching  behaviors  (e.g.,  falling  to  engage  a 
student  with  a  follow-up  probe  after  a  correct  but  hesitant  response)  can  be  observed.  Although 
the  film  emphasizes  “problem-  behaviors,  It  also  offers  the  opportunity  to  discuss.  If  not 
observe,  effective  as  well  as  Ineffective  teaching  behavior.  Generally,  the  film  Is  used  to 
Introduce  Form  281,  after  which  the  Items  on  a  Form  281  relevant  to  the  film  are  completed  by  each 
member  of  the  class  and  then  discussed.  Although  estimates  vary  from  base  to  base,  the  time 

devoted  to  the  use  of  Form  281  In  the  Instructor  supervisor's  course  Is  about  1  to  3  hours. 

Recent  graduates  of  the  supervisor  training  course  Interviewed  tended  to  report  that 

Instruction  in  the  use  of  ATC  Form  281  comprises  an  acquaintance  with  the  rules  and  regulations 
pertaining  to  when  and  how  to  complete  It.  Generally,  however.  It  was  reported  that  this 

Instruction  excluded  guidelines  for  completing  the  comment  portion  of  Form  281;  Instruction  on  how 
to  observe,  mentally  take  notes  of  and  record  ongoing  behavior;  and  Instruction  on  how  to  conduct 
the  post-observation  briefing  In  which  specific  strengths  and  weaknesses  should  be  diagnosed  and 
suggestions  for  remediation  provided. 

Both  Instructors  and  supervisors  reported  considerable  latitude  In  how  ATC  Form  281  was  being 
used  to  provide  Instructor  feedback.  Therefore,  the  following  recommendations  are  made: 

1.  The  time  devoted  to  ATC  Form  281  In  the  Instructor  supervisor  course  should  be  upgraded  to 
a  highly  organized  simulation-intensive  block  of  Instruction. 

2.  Brief  training  films  simulating  effective  and  Ineffective  teaching  behaviors  should  be 
produced  for  each  of  the  12  Instructional  areas  on  ATC  Form  281.  Each  file  would  focus 
specifically  on  a  designated  area  and  Its  subcomponents,  as  noted  on  ATC  Form  281.  These  areas 
are: 

Preparation 
Content/structure 
Presentation  skills 
Communication  skills 
Demonstration  skills 
Use  of  training  alds/materlal  s 
Question/answer  techniques 
Management 

Supervising  group  activity 
Measurement 
Personal  Qualities 
Other 


Supervisory  trainees  would  be  expected  to  learn  how  to  accurately  observe  and  record  on  ATC 
Form  281  effective  and  ineffective  teaching  behaviors  in  each  of  these  areas. 

1.  Examples  of  diagnostically  relevant  and  Irrelevant  comments  and  sources  of  remediation 
pertaining  to  each  Item  on  ATC  Form  281  should  be  provided  during  the  Instructor  supervisor  course 
and  made  available  to  all  In-service  supervisors. 


2.  A  role-playing  performance  exercise  should  be  built  Into  the  supervisor  training  course  In 
which  each  student  conducts  a  post-observation  briefing  with  a  peer,  using  comments  from  a 
completed  ATC  Form  281.  Instructional  emphasis  should  be  placed  on  the  diagnostic  accuracy  of  the 
comments  made,  their  potential  for  constructive  feedback  to  the  Instructor,  and  the  specificity  of 
the  suggestions  made  for  remediation.  The  exercise  should  be  graded  and  an  appropriate  behavioral 
checksheet  prepared  for  student  evaluation. 

3.  A  manual  to  accompany  ATC  Form  281  should  be  prepared  which  defines,  provides  examples  of 
and  explains  the  rationale  for  each  of  the  behaviors  on  Form  281.  The  purpose  of  the  manual  would 
be  to  standardize  the  meanings  of  each  of  the  terms  used  on  the  form,  providing  a  common  reference 
and  Interpretation  for  all  Instructor  supervisors.  This  product  would  extend  but  also 
considerably  supersede  In  scope  and  update  In  accuracy  ATCR  52-8,  attachment  5,  "Standards  for 
Classroom  Instruction  to  Assist  Instructors  In  Using  ATC  Form  281." 

Approximately  75  man-days  are  estimated  for  completing  these  recommended  actions,  exclusive  of 
pilot-testing  and  media  production. 


Evaluation  of  Laboratory-  and  Field-Based  Instruction 

Although  ATC  Form  281  tends  to  emphasize  formal  classroom  Instruction,  a  large  percentage  of 
Air  Force  technical  training  Is  conducted  In  the  laboratory  or  field.  Approximately  50X  of  the 
Instruction  observed  at  Lackland  (e.g.,  security  police  specialist  course),  Keesler  (e.g.,  air 
traffic  control  specialist  course),  Sheppard  (e.g.,  electrician  specialist  course),  and  Lowry 
(e.g.,  F-l 5  aircraft  armament  systems  specialist  course)  depended  heavily  on  laboratory,  field  and 
simulation  experience.  Instructors  and  instructor  supervisors  reported  that.  In  these 
Instructional  settings,  ATC  Form  281  tends  to  be  less  useful  as  a  feedback  device  than  when 
Instruction  takes  place  In  the  formal  classroom.  Yet  some  Instructor  supervisors  must  use  ATC 
Form  281  In  these  settings  In  order  to  meet  the  quarterly  and  30-day  evaluation  requirements  for 
their  new  (1  to  3  months)  and  relatively  new  (3  to  12  months)  Instructors. 

For  most  supervisors  this  transfer  to  laboratory  or  field  use  Is  made  by  checking  “NA"  f‘'ot 
Applicable)  for  Items  that  are  Irrelevant.  Most  supervisors  report  trying  to  avoid  using  Form  -31 
In  the  field  or  laboratory  when  possible,  but  other  supervisors  report  extensive  use  of  Form  281 
in  these  settings  when  7QX  or  80S  of  their  course  Is  conducted  on  the  flightline.  In  the  hangar, 
or  field  shop.  Supervisors  report  several  problems  when  Forms  281  are  used  exclusively  during 
formal  classroom  Instruction  In  a  course  that  Is  primarily  field-  or  laboratory-dominated.  When 
Forms  281  are  used  exclusively  during  formal  classroom  instruction  In  a  field-  or 
laboratory-dominated  course,  feedback  specific  to  the  field  setting  does  not  occur,  even  though  a 
large  percentage  of  the  instruction  takes  place  In  these  settings.  Conversely,  when  Forms  281 
must  be  used  consistently  In  a  field  setting,  sometimes  a  large  number  of  Items  must  be  marked 
"NA"  or  else  be  Interpreted  broadly  to  fit  the  field  setting,  leaving  some  behaviors  unevaluated 
or  Interpreted  to  mean  something  different  for  a  particular  supervisor  at  a  particular  training 
center.  Therefore,  the  following  recomnendatlon  Is  made: 

1.  Each  technical  area  of  training  at  the  Wing,  specialty  or  course  level  should  be 
encouraged  to  develop  a  field-  or  laboratory-based  counterpart  to  ATC  Form  281,  when  applicable, 
which  uniquely  captures  the  performance,  demonstration,  and  activity  skills  related  to  that 
setting.  This  form  could  exhibit  one  section  of  Items  common  to  all  areas  of  field-  or 
laboratory-based  training  In  the  Air  Force  and  another  section  unique  to  the  Individual  Wing, 
specialty  or  course.  To  accomplish  this,  a  field-  or  laboratory-based  counterpart  to  Form  281 
could  be  developed  in  two  or  three  specialty  areas,  thereby  beginning  the  development  of  a  core  of 
items  common  to  all  specialties.  These  instruments  could  then  be  used  as  prototypes  for  other 
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Wings,  specialties  or  courses  In  constructing  their  own  sets  of  unique  Items,  where  such  a  form 
may  be  needed.  One  approach  for  transferring  the  needed  Instrument  development  skills  to  other 
specialties  would  be  to  conduct  a  brief  workshop  for  a  master  Instructor  or  supervisor  from  each 
specialty  area  Interested  In  developing  a  field-based  evaluation  form.  The  workshop  would 
Instruct  participants  on  the  set  of  core  Items  already  developed  and  how  to  phrase,  scale  and 
score  Items  consistent  with  the  objectives  of  their  own  specialty  area.  The  workshop  would 
constitute  a  brief  but  practical  hands-on  Introduction  to  the  construction  and  validation  of  a 
simple  field-  or  laboratory -based  observation  Instrument  emphasizing  feedback  to  the  Instructor. 
Some  of  the  content  presented  In  Section  V  of  this  report  could  be  relevant  for  such  Instruction. 

Approximately  100  man-days  are  estimated  for  completing  this  recommendation,  exclusive  of 
pilot-testing. 


Instructor  Peer  Evaluation 


Peer  evaluations  were  regularly  conducted  in  approximately  25S  of  the  technical  courses 
observed  by  the  contractor.  In  each  instance  In  which  peer  evaluations  consistently  occurred, 
they  were  rated  by  Instructors  as  the  most  valuable  source  of  feedback  they  received — more 
valuable  than  ATC  Form  281  and  the  post-observation  debriefing.  As  noted  earlier  In  this  report, 
peer  evaluations,  although  consistently  used  In  some  Instructional  environments,  are  often 
conducted  Informally  at  the  Invitation  of  a  particular  Instructor,  sometimes  using  ATC  Form  281  as 
a  general  guide  or  starting  point  for  the  observation.  However,  these  peer  evaluators  tended  to 
supplement  Items  on  ATC  Form  281  with  additional  Indicators  of  teaching  effectiveness  that  tended 
to  "personalize"  ATC  Form  281  down  to  the  specific  specialty  being  trained  for,  by  looking  for  how 
a  particular  Instructor  handled  complex  or  problematic  portions  of  specific  areas  of  the 
curriculum.  The  concept  of  peer  evaluation  among  those  who  used  It  appeared  to  be  well  accepted 
and  valued  as  a  meaningful  source  of  Instructional  feedback.  Contrary  to  Instructor  supervisors 
who  already  may  be  overburdened  with  the  required  administration  of  ATC  Form  281,  peers  may  be  a 
possible  source  of  additional  feedback  which  could  supplement  that  provided  by  Form  281.  Since 
ATC  Form  281  Is  seen  by  some  Instructors  and  used  by  some  Instructor  supervisors  as  primarily  a 
tool  for  evaluation  not  feedback,  the  following  recommendation  Is  made: 

Section  II  of  this  paper  identified  28  distinct  Indicators  of  teaching  effectiveness  under  the 
more  general  teaching  dimensions  of  clarity,  variety,  task  orientation,  student  engagement  in  the 
learning  process,  and  success  rate.  These  Indicators,  along  with  their  behavioral  descriptors 
(provided  in  Tables  1,  2,  3,  6,  7),  could  be  placed  in  the  form  of  an  observation  instrument 
suitable  for  use  by  peers.  In  Section  V  of  this  paper  a  review  of  possible  measurement  formats 
Indicated  that  the  Checklist  and  Likert  Scales  represented  the  most  practical  formats  for 
providing  feedback  to  the  ATC  Instructor  when  the  criteria  of  acceptability  to  the  user,  cost, 
objectivity,  increased  feedback  provided,  ease  of  Implementation,  validity,  and  reliability  were 
considered.  An  observation  Instrument  suitable  for  peer  use  using  a  Checklist,  or  combination  of 
Checklist  and  Likert  Scale,  and  Incorporating  the  teaching  effectiveness  indicators  derived  from 
the  research  literature,  as  described  In  Section  II,  could  constitute  a  new  feedback  initiative. 
Instructors  could  be  required  to  complete  and  document  a  specific  number  of  peer  evaluations  each 
year,  regardless  of  Instructor  status,  the  results  of  which  could  be  kept  confidential,  if  so 
desl red. 

Approximately  80  man-days  are  estimated  for  completing  this  recommendation. 


Student  Evaluation  of  Instructors 


ATC  Form  736  is  commonly  used  by  either  individual  students  or  a  group  of  students  to  comment 
on  an  instructor’s  performance  during  a  course.  As  was  noted  earlier,  few  individual  student 
critiques  are  actually  completed,  although  some  variation  across  courses  and  instructors  was 
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noted.  Also,  some  group  or  class  critiques  provide  only  general,  congratulatory  or 
problem-specific  comments  that  are  difficult  to  trace  back  to  specific  Indices  of  teaching 
effectiveness,  thereby  limiting  the  diagnostic  value  of  the  data.  Although  the  open-ended 
response  format  provided  by  ATC  Form  736  encourages  originality  and  spontaneity,  the  exclusive  use 
of  such  a  format  may  also  direct  student  responses  away  from  more  structured  indices  of  teaching 
effectl veness,  such  as  those  identified  In  Section  II  of  this  paper.  Therefore,  the  following 
reconnendatlon  Is  made: 

In  addition  to  ATC  Form  736,  a  Likert-type  4-  or  5-polnt  scale  could  be  developed  to  reflect 
the  same  28  Indices  of  teaching  effectiveness  (Identified  In  Section  II)  and  earlier  recommended 
for  use  In  a  peer  evaluation  form.  In  this  manner,  student  responses  could  complement  and  extend 
the  feedback  provided  on  the  peer  Instructor  evaluation  form,  since  both  forms  would  contain  the 
same  or  similar  indices  of  teaching  effectiveness.  Since  the  peer  evaluation  form  was  proposed 
for  feedback  to  the  Instructor,  the  addition  of  a  matching  structured  student  response  form  could 
be  a  useful  complement  to  It.  Such  an  Instrument  could  be  made  machlne-scorable  and  require  less 
than  20  minutes  of  class  time  to  complete.  If  a  Likert  format  Is  to  be  used,  the  number  of 
student  responses  for  each  Item  could  be  provided  in  percentage  and  bar  graph  form  and  Item 
averages  could  be  compared  to  a  normative  sample  of  Instructors  from  that  content  area,  specialty 
or  Ming.  Although  the  primary  purpose  of  a  structured  student  response  form  would  be  to  provide 
feedback  to  the  Instructor  on  standardized  Indices  of  teaching  effective-  it  could  also  become 
part  of  the  Instructor's  permanent  record. 

Approximately  80  man-days  are  estimated  for  completing  this  recommendation,  exclusive  of 
pilot-testing. 


Technical  Training  Instructor's  Course 

The  Technical  Training  Instructor's  Course  fs  a  fully  exportable  instructional  package  that 
can  be  presented  at  any  Technical  Training  Center.  All  new  Instructors  must  attend  the  course 
prior  to  their  first  teaching  assignment.  Normally,  the  course  is  taught  In  approximately  a 
3-week  period.  It  has  recently  been  revised  and  Is  now  undergoing  pilot-testing  at  Sheppard  AFB. 

The  course  teaches  effective  communication,  the  psychology  of  teaching  and  learning.  Air  Force 
Instructor  rules  and  regulations,  and  how  to  prepare  and  present  lessons,  among  many  other 
topics.  Conspicuously  missing  from  this  curriculum,  however.  Is  reference  to  the  teaching 

effectiveness  Indicators  Identified  In  Section  II  of  this  paper.  Therefore,  the  following 
recommendations  are  made: 

1.  The  Technical  Training  Instructor  Course  provides  the  logical  place  to  introduce  new 

Instructors  to  the  effective  teaching  behaviors  Identified  In  Section  II.  It  Is  recommended  that 
1  hour  of  Instruction  for  each  of  these  five  effective  teaching  behaviors  be  added  to  the 

Technical  Training  Instructor's  Course.  This  would  include  five  lectures  that  would  (a)  Introduce 
the  general  behavior  and  Its  Individual  components,  (b)  provide  a  prescriptive  rationale  for  why 
the  particular  behavior  Is  important  for  effective  teaching,  and  (c)  Illustrate  some  specific 
Instructional  practices  by  which  the  behavior  can  be  satisfactorily  exhibited  In  the  classroom 
(e.g..  Tables  1,  2,  3,  6,  7).  Section  II  of  this  paper  lays  the  foundation  for  these  five 
lectures.  In  addition.  It  Is  recomnended  that  approximately  3  hours  of  exercises  and  testing 
accompany  the  5  hours  of  lecture  to  acquaint  students  with  the  application  of  these  concepts  In 
the  real  world  of  the  classroom. 

2.  Although  already  suggested  as  an  addition  to  the  supervisor  training  course.  It  is 

recommended  that  a  manual  be  prepared  that  provides  standardized  definitions  and  examples  for  the 
behaviors  identified  on  ATC  Form  281.  This  manual  would  be  equally  useful  as  a  pedagogical  tool 
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In  the  Instructor  training  course  for  Introducing  the  specific  kinds  of  behavior  to  be  observed  by 
the  Instructor  supervisor  using  ATC  Form  281.  As  noted  earlier,  801  of  the  teaching  effectiveness 
Indicators  Identified  In  Section  II  of  this  paper  currently  have  counterparts  on  ATC  Form  281, 
thereby  providing  a  high  degree  of  Integration  between  this  and  the  above  recommendation. 

Approximately  120  man-days  are  estimated  for  completing  this  recommendation,  exclusive  of 
pilot- testing. 
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APPENDIX  A:  SOME  INSTRUCTOR  EVALUATION  FORMS  USED  IN  PUBLIC  SCHOOLS 
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(Specimen  1) 

CRITERIA  AND  INDICATORS  SAMPLE- - 
Teacher  Performance  Evaluation 


CRITERION  1 

IA.  The  teacher  demonstrates  effective  planning  skills. 

IB.  Plans  for  Teaching. 

IC.  Plans  for  Instruction 


INDICATORS :  1 . 

2. 


3. 


4. 


5. 


6. 

7. 


8. 


9. 


10. 


11. 

12. 


13. 


14. 


Selects  appropriate  long-range  goals. 

Writes  instructional  objectives  that  are  related 
to  long-range  goals. 

Selects  objectives  at  the  correct  level  of  diffi¬ 
culty  to  assure  successful  learning  experiences 
for  each  student. 

Includes  teaching  methods  and  procedures  relevant 
to  the  objective. 

Includes  relevant  student  activities. 

Utilizes  both  formative  and  summative  evaluation 
procedures . 

Plans  appropriate  time  allotment. 

Selects  a  variety  of  teaching  methods  and  proce¬ 
dures  along  with  a  variety  of  student  activities 
to  use. 

Writes  appropriate  instructional  objectives. 

Selects  objectives  congruent  with  district 
curriculum. 

Incorporates  appropriate  materials  and  resources. 

Develops  appropriate  instructional  objectives. 

Writes  plans  which  are  timely,  current  and  readily 
available. 

Follows  district  curriculum  guides. 
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CRITERIA  AND  INDICATORS  SAMPLE- - 
Teacher  Performance  Evaluation 

CRITERION  2 

2A.  The  teacher  implements  the  lesson  plan. 
2B.  Implements  an  instructional  plan. 

2C.  Implements  a  lesson. 

2D.  Implements  a  lesson  design. 


INDICATORS:  1.  Reviews  and  previews;  provides  the  structure  of 

learning. 

2.  States  instructional  objectives. 

3.  Provides  input  related  to  objectives. 

4.  Models  activities  congruent  with  topic  being  taught 
and  provides  guided  practice  to  reinforce  concepts. 

5.  Utilizes  lesson  summary  techniques. 

6.  Provides  independent  practice  activities. 

7.  Indicates  positive  directions  for  moving  from  one 
activity  to  the  next. 

8.  Checks  for  understanding. 

9.  Provides  appropriate  lesson  flow. 

10.  Uses  appropriate  methods. 

11.  Uses  appropriate  teaching  behaviors. 

12.  Uses  methods  and  teaching  behaviors  effectively. 

13.  Makes  use  of  appropriate  timing. 

14.  Reviews,  previews,  and  verbalizes  the  purpose  of 
the  lesson. 

15.  Presents  concepts  in  an  orderly  manner. 

16.  Demonstrates  knowledge  of  facts  and  concepts  of 
the  specific  lessen. 

17.  Presents  appropriate  background  inf ormatic n . 

18.  Uses  appropriate  questioning  techniques. 
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Criteria  and  Indicators  Sample- -Teacher  Performance  Evaluation 
CRITERION  2  -  Continued 

19.  Makes  effective  use  of  available  instructional 
media  and  material. 

20.  Checks  for  students'  understanding  before  going 
on  with  the  lesson. 

21.  Provides  for  guided  practice  (where  appropriate). 

22.  Presents  instruction  at  an  appropriate  level  of 
difficulty . 
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CRITERIA  AND  INDICATORS  SAMPLE- - 
Teacher  Performance  Evaluation 


CRITERION  3 


3A.  The  teacher  motivates  students. 


INDICATORS:  1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 


11. 


12. 


13. 


14. 


15. 


Communicates  challenging  scholastic  expectations 
to  students. 

Responds  positively  to  students. 

Stimulates  students  by  choosing  proper  materials 
and  techniques. 

Gives  feedback  to  students. 

Uses  methods  to  stimulate  creative  expression. 
Stimulates  creative  thinking. 

Promotes  active  participation  during  the  lesson. 

Communicates  expectations  to  students. 

Moves  about  the  classroom  to  facilitate  learning. 

Provides  opportunities  for  students  to  experience 
success . 

Stimulates  student  interest  by  utilizing  appro¬ 
priate  content. 

Provides  variety  in  methods  and  materials  through¬ 
out  the  year. 

Sustains  student  attention  and  responses  by  plan¬ 
ning  activities  at  the  appropriate  difficulty 
level . 

Provides  prompt  feedback  to  students. 

Shows  enthusiasm  through  teaching  methods  and 
behaviors . 
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HIGH  ACHIEVEMENT  COMMUNITY  SCHOOLS 


Teacher's  Name 


Building 


Grade/Subject 


IRECTIONS:  For  each  descriptor,  place  a  check  in  the  column  that  best  describes  the 

teacher’s  performance  for  that  item.  After  assessing  the  teacher's 
performance  for  each  descriptor,  please  assess  performance  for  the 
criterion  by  placing  a  check  in  the  appropriate  column. 

Please  check  "Target  for  Growth"  for  those  descriptors  and  criteria 
which  the  teacher  should  focus  on  for  professional  growth. 

Specific  descriptive  comments  must  accompany  any  assessment  where  perfor¬ 
mance  does  not  meet  district  standards  and  should  also  supplement  "Target 
for  Growth"  and  other  assessments. 


The  performance  review  should  be  used  to  provide  the  teacher  with  a 
summatlve  performance  assessment  and  an  opportunity  to  respond  to  the 
summatlve  evaluation.  Please  examine  performance  in  each  criterion  and 
check  the  term  which  represents  a  summatlve  evaluation  of  the  teacher’s 
performance. 


XPLANATION  OF  THE  SCALE: 

NOT  RATED:  Insufficient  data  for  rating  performance. 

MUST  IMPROVE:  Performance  jeopardizes  continued  employment  in  the 
district. 

NEEDS  IMPROVEMENT:  Performance  is  below  the  high  standard  required. 
MEETS  STANDARD:  Performance  meets  district  standards. 

EXEMPLARY:  Performance  exceeds  district  standards. 

TARGET  FOR  GROWTH:  Performance  area(s)  in  which  improvement  is  deemed 

most  beneficial. 
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CRITERION:  Plans  for  instruction 

(long  and  short  term) 


].  Develops  appropriate  instructional 
objectives. 


o 

z 


a 

LU 

2 


</> 


u 

> 

a 

ec 


§5 


w 

> 

o 

ae 


2.  Includes  methods,  procedures,  and 
student  activities  relevant  to 
objectives. 

3.  Develops  both  formative  and 
summative  evaluation  procedures. 

4.  Writes  plans  which  are  timely, 
current,  and  readily  available. 


5 

g 

U) 

X 
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CRITERION:  Plans  for  instruction 

(long  and  short  term) 


COMMENTS: 


TARCET 

FOR 

GROWTH 


TARCET 

FOR 

GROWTH 
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TARCET 

FOR 

CROW7H 


iKiTERION:  Fulfills  professional 

responsibi lit  ies. 


I.  Has  regular  attendance  and  is 
punctual . 

Provides  and  maintains  accurate  records. 

3.  Assumes,  completes  duties  promptly  and 
effectively. 

*•  •  Handles  confidential  matters  or 

information  in  a  professional  manner. 

5.  Maintains  good  grooming. 

o.  Assumes  school  related  student 

management  reeponelbllitles  outside  of 
classroom. 

7.  Works  with  and  through  organization 
ladder  when  solving  problems  or 
seeking  change. 

8.  Adheres  to  school  regulations  and  board 
policies. 


CRITERION:  Fulfills  professional  responsibilities. 
COMMENTS: 
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MEETS  Dl STRICT 
STANDARD 


Subject  or  Grade _ _ _  Date 


(Specimen  2) 
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Plan  ot  Action/Improvements: 


(Specimen  3) 


OBSERVATION  RECORD  FORM 
COVINGTON  PUBLIC  SCHOOLS 


Teacher  Observed: 


Gets: 


Observer: _ 

I.  Planning  lor  Instruction: 


Component  1.1 
Component  1.2 

Component  1.2 
Component  1.4 


Describe*  appropriate  student  objectives  and  teaching  procedures  to  be  used  for  each  objective 

Develops/selects  procedures  or  materials  that  provide  students  with  expropriate  practice  of 
application  on  objectives. 

Describes  procedures  and  materials  for  assessing  learner  progress  on  lesson  objectives. 

Develops  and  maintains  a  record  of  individual  student  progress. 


COMMENT 


Implementing  Instruction: 

Component  2.1  Uses  techniques  to  develop  student  interest  and  involvement  in  lesson. 

Component  2-2  Uses  procedures  and  materials  which  accomodate  differences  in  the  learning  needs  of  students 

Uses  instructional  activities  which  provide  students  with  the  opportunity  to  be  actively  involved 
in  the  lesson. 


Component  2-3 


Component  2.4 
Component  2-5 


Provides  students  with  the  opportunity  for  appropriate  practice  or  application  on  objectives 
(including  homework). 

Monitors  student  progress  or  performance  during  lesson  to  identify  the  need  for  clarification, 
assistance,  feedback  or  lesson  modification. 


Component  2.4 


Provides  an  orderly  and  attractive  learning  environment. 


COMMENT 


m. 


Managing  the  Instructional  Environment: 

it  2.1  Communicates  expectations  for  student  behavior. 

Manages  routine  tasks  efficiently. 

Uses  appropriate  techniques  to  involve  students  who  are  off  task. 

Takes  appropriate  actions  when  disruptions  interfere  with  learning  of  other  students 
Provides  assistance  to  students  having  difficulty. 

Reinforces  and  encourages  the  efforts  of  students  to  maintain  involvement, 
b  courteous  and  pleasant  with  students. 

Uses  instructional  time  efficiently. 


Component  3.2 
Component  3-3 
Component  3.4 
st  3J 
3.4 
«  3.7 
it  3.S 


COMMENT. 


IV.  Communicating  with  Students: 

'  Component  4.1  Communicates  instructional  plan,  lesson  content  and  activities  in  a  logical  sequence. 

"  Component  4.2  Uaes  effective  and  appropriate  vocabulary  and  sentence  structure  during  oral  instruction. 

— —  Component  4-3  Communicates  an  accurate  command  of  subject  matter,  clarifying  information,  explanations  and 
directi  ons. 

.  -  Component  4.4  Uaes  effective  written  communication, 
i-  Component  4.3  Encmrages  atudent  expression  of  ideas,  feelings,  and  opinions. 

-  Component  4.4  Communicates  enthusiasm  and  high  expectations  for  teaching  and  learning. 

COMMENT _  _ 


**  (This  form  u  to  be  used  to  record  teething  performances  observed  during  §  particular  ciassroom  out/.  It  is  not  s  complete  evaluation  / 
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(Specimen  4) 


COMPETENCY  I  : 
Indicator  1. 
Indicator  2. 
Indicator  3. 

Indicator  . 

Indicator  5. 
COMPETENCY  II: 

Indicator  6. 

Indicator  7. 

Indicator  8. 

COMPETENCY  III: 

Indicator  9. 

Indicator  10. 

COMPETENCY  IV: 

Indicator  11. 
Indicator  12. 

COMPETENCY  V: 

Indicator  13  . 

Indicator  in. 


MISSISSIPPI  TEACHING  COMPETENCIES  AND  INDICATORS 
PLANS  INSTRUCTION  TO  ACHIEVE  SELECTED  OBJECTIVES. 

Specifies  or  selects  learner  objectives  for  lessons. 

Specifies  or  selects  teaching  procedures  for  lessens. 

Specifies  or  selects  content,  materials,  and  media  for 

lessons . 

Specifies  or  selects  materials  and  procedures  lor  assessing 
learner  progress  on  the  objectives. 

Plans  Instruction  at  a  variety  of  levels. 

ORGANIZES  INSTRUCTION  TO  TAKE  INTO  ACCOUNT  INDIVIDUAL 
DIFFERENCES  AMONG  LEARNERS. 

Organizes  instruction  to  take  Into  account  d l f fercr-cea  meng 
learners  in  tnelr  capabilities. 

Organizes  Instruction  to  take  Into  account  differences  among 
learners  in  their  learning  styles. 

Organizes  Instruction  to  take  into  account  differences  among 
learners  in  their  rates  of  -earning. 

OBTAINS  AND  USES  INFORMATION  ABOUT  THE  NEEDS  AND  PROGRESS  OF 
INDIVIDUAL  LEARNERS. 

Uses  teacher-made  or  teacher-selected  evaluation  materials  or 
procedures  to  obtain  information  about  learner  progress- 

Communicates  *lth  individual  learners  about  their  needs  and 
progress . 

OBTAINS  AND  USES  INFORMATION  ABOUT  THE  EFFECTIVENESS  OF 
INSTRUCTION  TO  REVISE  IT  WHEN  NECESSARY. 

Obtains  information  on  the  effectiveness  of  lnstructl-n. 

Revises  instruction  as  needed  aging  evaluation  results  and 
observation  data. 

USES  INSTRUCTIONAL  TECHNIQUES.  METHODS .  AND  -EDI  A  RELATE!  ~0 
THE  OBJECTIVES. 

Uses  teaching  methods  appropriate  for  adjectives,  learners  and 
environment . 

Uses  instructional  equipment  and  other  instructional  aids. 


85 


•.  tatcr  15. 

MF.-ITfNCY  VI: 
Indicator  16. 

: icator  17 . 

Indicator  18 . 
Indicator  19. 
Indicator  20. 
COMPETENCY  VII: 
Indicator  21. 
Indicator  22. 

Indicator  23- 

COMPETENCY  VIII: 
Indicator  21*. 

'odicator  25- 
Indicator  26 . 
Indicator  27. 

'  ETENCY  IX: 

'-itcacor  23. 

‘ n : i c  a  t  a  r  29. 
‘••r  ETENCY  X: 

Ucator  ;0. 

■  i :  :  a  t  o  r  3 1  . 


Vses  instructional  materials  that  provide  learners  with 
appropriate  practice  on  objectives. 

COMMUNICATES  WITH  LEARNERS. 

Gives  directions  and  explanations  related  to  lesson  content. 

Clarifies  directions  and  explanations  when  learners 
misunderstand  lesson  content. 

Uses  responses  and  questions  from  learners  in  teaching. 

Provides  feedback  to  learners  throughout  the  lesson. 

Uses  acceptable  written  and  oral  expression  with  learners. 

DEMONSTRATES  A  REPERTOIRE  OP  TEACHING  METHODS. 

Implements  learning  activities  in  a  logical  sequence. 

Demonstrates  ability  to  conduct  lessons  using  a  variety  of 
teaching  methods. 

Demonstrates  ability  to  work  with  individuals,  3mall  groups, 
and  large  groups. 

REINFORCES  AND  ENCOURAGES  LEARNER  INVOLVEMENT  IN  INSTRUCTION. 

Uses  procedures  which  get  learners  initially  involved  in 
lessons . 

Provides  learners  with  opportunities  for  participating. 
Maintains  learner  involvement  in  lessons. 

Reinforces  and  encourages  the  efforts  of  learners  to  maintain 
involvement . 

DEMONSTRATES  AN  UN I S3£7A.\ DING  OF  THE  SCHOOL  SUBJECT  BEING 
TAUGHT  AND  DEMONSTRATES  ITS  RELEVANCE . 

Helps  learners  neccgntce  the  purpose  and  iaporta: oe  vf  topics 
or  activities. 

Demonstrates  knowledge  in  the  subject  area. 

ORGANIZES  TIME.  SF-.IE.  MATERIALS  ,  AND  EQUIPMENT  - 
INSTRUCTION . 

Attends  to  routine  :a.v<s, 

Uses  instructional  tone  e  r  ec  t  ively  . 
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Indicator  32. 

COMPETENCY  XI: 

Indicator  35- 
Indicator  28. 

Indicator  27. 

Indicator  29. 
Indicator  31* 
COMPETENCY  XII: 

Indicator  33- 
Indicator  314- 
Indicator  35- 
COMPETENCY  XIII: 
Indicator  36. 
Indicator  37. 

Indicator  38. 
COMPETENCY  XIV: 
Indicator  39. 
Indicator  *<0. 
Indicator  . 

I nd lcator  «2 . 


Provides  a  learning  environment  that  la  attractive  and 
orderly . 

DEMONSTRATES  HIGH  EXPECTATIONS  FOR  LEARNERS'  ACADEMIC 
PERFORMANCE. 

Conveya  the  impression  of  knowing  what  to  do  and  how  to  do  It 

Helps  learners  recognize  the  purpose  and  importance  of  topics 
or  activities. 

Reinforces  and  encourages  the  efforts  of  learners  to  maintain 
involvement . 

Provides  feedback  to  learners  throughout  the  lesson. 

Uses  instructional  time  efficiently. 

DEMONSTRATES  ENTHUSIASM  FOR  TEACHING  AND  LEARNING  AND  THE 
SUBJECT  BEINC  TAUGHT. 

Communicates  personal  enthusiasm. 

Stimulates  learner  interest. 

Conveys  the  impression  of  knowing  what  to  do  and  now  to  do  it 

HELPS  LEARNERS  DEVELOP  POSITIVE  SELF-CONCEPTS. 

Demonstrates  warmth  and  friendliness. 

Demonstrates  sensitivity  to  the  needs  and  feelings  of 
learners . 

Demonstrates  patience,  empathy,  and  understanding. 

MANAGES  CLASSROOM  INTERACTIONS. 

Provides  feedback  to  learners  about  their  behavior. 

Promotes  comfortable  interpersonal  relationships. 

Maintains  appropriate  classroom  behavior. 

Manages  disruptive  behavior  among  learners. 
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All  learners  hive  opportunity  fot  active 
participation  in  two  or  «dr4  Activities. 


(Specimen  5) 

_ OBSERVATION  RECORD  _ 

>  “  Oat* 

✓  EVALUATION  RECORD  10-2. 2. -8 A 


Scnooi  O' strict 


Assignment/Grad# 


-  iranaJ  1  hr  2 


Co.-Oist.  No 


^“■servation  Oat* 


scheduled 


Nam*  of  Appraiser 


Unscheduled 


Seginning  Tim* 


Scnooi  rear  t 


_  Taacnar  Supervisor  o^OtnerAporais 

C  Onai 

Ending  Tim*  U:SO 


Subject  Areals)  Observed 


PC 


Qlrecttone;  Circle  me  numoens)  lor  eacn  indicator 
ixpectanon  "  oenaviors  and  nas  a  credit  value  at 
M.  Column  EQ"  represents  Exceptional  Quail. ; 
we  given  m  Column  "SE."  Exceptional  Quality  credit  i 
Provide  documentation  when  marking  either  Column  " 
nd  tn*  second  appraiser  will  compute  credits  within  7  day 


umn  will  be  marked  lor  eacn  indicator.  Column  A/8E”  is  used  to  note  Aosent/Seiow 
represents  tn*  'Standard  Expectation"  lor  eacn  indicator  anc  nas  a  credit  vaiue 
indicator  and  has  a  credit  value  of  i  To  give  credit  to  Column  EQ.  credit  must 
I*  to  some  indicators.  For  these,  a  hyphen  has  been  placed  m  Column  EQ. 
The  teacher  supervisor  will  compute  credits  at  tn*  end  cf  the  aopraisai  period, 
formal  observation.  Corrections  should  be  initialed  by  person  filling  out  me  *orm 


Instructional  Strategies 


Columns 


Provides  opportunities  for  students  to  participate 
and  successfully. 


vanes  activities 
nteracts  with  students 
solicits  participation 
extends 
provides  time 


laments  at  appropriate  iev*i  of 


2.  Evaluates  and  provides  feedback  on  student  orogress  dur¬ 
ing  instruction. 


oommunicaies  excectanons 


soiicts  'esponses  ‘or  nssessm?~' 
-em'orces 

prcvdss  correc' ve  'oeoDac* 
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"OR  EVALUA"  cn  recor 
CCMAin  CRECi'  "QrAL 


II.  Classroom  Management  and  Organization 


Columns 

A/8E  SC  SO 


3  Orga-  2« s  mate-ais  and  stuce'vs 


4  .Maximizes  amount  of  time  availade  for  instruction. 


5.  Manages  student  behavior. 


SSCU'CS  StuOSr‘  a"»-ViC" 

jses  procedures  *3.:. nes 
gives  sommist'a-  ,e  jirectc'i 
uses  seating/g-;.c  -g 
nas  m*i#nais/a  cs  'acini  es  -»ir. 


oegms/enas 

impi •moms  seouence  of  ac:<v  ties 
maintains  pace 
maintains  focus 
Keeps  students  engaged 


specifies  expectations 
prevents  off-tasx  oenavior 
redirects  off-tasx  penavior 
stops  inappropriate  benavior 
stops  disruptive  oenavior 
applies  rules 

reinforces  appropriate  oenavior 


O*  EVALUATION  RECORD 
OOMAIN  CREDIT  TOTAL 
(SE  credits  *  EQ  credits) 


11 


ill.  Presentation  of  Subject  Matter 


6.  Teaches  for  cognitive,  affective,  and/or  psychomotor  learn¬ 
ing  and  transfer 


mg  and  transfer.  .  ,  ,  la- 

PmcMtlA  baulA  fa t‘ 


K. 


a.  oegms  *«th  introduction 

b.  uses  content  sequence 

c.  relates  prior/future  learning 

d.  defines/describes 

e.  elaooraies  critical  anribuies 
stresses  generamation/prirsc.: 
transfers 
closes  instruction 


ift  thg.m 

— : - .  rur\Aea.r  f 

rvejft  u^eeJc.1  u  ^/ffV teJ  dolor  CUtJl  s 

— OV  -Hte  jLshe.A  iWjjQLur  hnw\fc-.  n 

-We.  will  gYCLryvolc.^  of 
_ /tushes  rw*  u/a^'  » 

Darina  ciosina  4t>  look.  ftr 

t/iCLTyv^DltS  cf  pottery  ett  home.  OlaJL  in 


0 


18 
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ill.  Presentation  of  Subject  Matter  (continued) 


1  P'esents  , reformation  accurately  and  clearly 

ou  TufrW  XtuAjla 


tUffere/ct  I  rv^U^LA  Tri  be  s  7 


8.  Uses  acceptable  communication  skills 
tL.  Foult  errors  of  SuJajccJ 

r>itiu  I*  e  •  "  *TVj  /m  i  c 


i*.  Learning  Environment 

9  Uses  strategies  to  motivate  students  for  learning. 


i0  Maintains  suoportive  environment. 

<Xj JSL TH*-  -3orrie< 


~  neva 


a  makes  no  significant  errors 
Ly  3  uses  aoorconata  vocaouu'r 

c  c  sxoiains  ciaarty 

li  a  stresses  pomts/Oimensions 

I A  e  clarifies  misunderstanding 

-fJTd.1 

key 


a.  usee  correct  grammar 
Xnl?  pronounces  correctty/cieeny 
i*\X  c.  usee  accurate  language 

l  &  d.  demonstrates  written  skills 


FOR  EVALUATION  RECCRO 
OOMAIN  CREDIT  TOTAu 
(SE  credits  *  EO  credits) 


r states  to  •'••rests 

egj^es'zes  vaiue/imoons-:e 
•nfi^ts  efforts 


manges 


ranker  5eeJs-j  C OtftorvdbufK 

Speaks  +b  S'fujlivlb  i  n 

Km ”h3r\e_  0*P  v'Oiee. 


a  avoids  sarcasm/negative  :•  fcisn 
3  maintains  courteous  ciims:* 
c  encourages 


e  estaoiisnes  raooon 


FOR  EVALUATION  PECORO 
DOMAIN  CREDIT  nD'a. 
iSE  cedits  -  EO  res  -s. 


Columns 

A.ee  sc  £Q 


g  : 

0  0 

®  ' 

’ 
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V.  Growth  and  Responsibilities 


Column 

A/BE  SE  EC 


1 1  Plans  for  ana  engages  m  professional  development. 


a  grows  D'ofessonaity 

D  stays  cu"ent-content 

c  stays  Current-methods 


c 

0 


12.  Interacts  and  communicates  with  parents. 


13. 


Complies  with  policies,  operating 
requirements. 


14.  Promotes  and  evaluates  student  growth. 


a.  initiates  communications 
0.  conducts  contarancaa 
c.  raoons  prograss 
0.  maintains  confidentiality 


a.  follows  TEA  requirements 

b.  follows  distnct/campus  policies/ 
procedures 

C.  performs  assigned  duties 
d.  follows  promotion  procedures 


ipates  m  goal-setting 
instruction 
imams  progress 


W  reports  pr 


progress 


°  Q. 

0  Q 

0  (0 
0  Q) 


FOR  EVALUATION  RECORD 
DOMAIN  CREDIT  TOTAL 
(SE  credits  ♦  EO  credits) 


Comments: 


(The  signeture  of  me  (teener  indicates  tnst  ne/sne  nss  reviewed  ano  received  a  copy  of  tnis  record  i 

to+LdL  (L ppbAMO*.  &  /g-aj-SL  I/-3-S7 

Teacner  Signature/Oate  Received  eppraiser  Signaturs/Oate  Completed  Oat#  of  Cc^'eren;? 

Origmei  Copy— Central  Office  Ha'-. 

Cooy  *2— Teacher's  Supervisor 
Copy  «3— Teacner 
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Scnooi  Oismct 


Teacher  _ 

Assignment/Gradt 


Domain  Subtotal 

Teacner  *  Supervisor  e«-  — - . 

Instructions:  “’i's'e'  me  OOMA.\  CREC 


Co  -Out  Nc  2-55”  9  0  / 

?  U2 

^  Scnoc  »ss-  -9s-  •■ie’ 


Scnooi 


Oomsm 


-ist'jcrona  a:-ateges 
Classroom  Management  *no  C'ja-iza- 
presentation  ;*  SuOiect  Matt#' 
Learning  Env'Onment 
Growtn  ana  Responsiowties 


talS  'torn  ms  Evaluation  Recoro  Multiply  Apprusai  2  oy  60  to  ootair  rns  Oc~a  *  S-cca 
s  Appraisal  2  Domain  Suototai 

IU  J L  .  6C_Ai> 

A  .  e:-J J£jtf 

Z3L  -2 31  '  6  Z.JMJ4 

If  1 7  x  so  *  1 0>  2./ 

fla  3  Q  x  60 »  J  2u.  0 


_JA 


JIl-  . 

zaS:  . 


Other  Appraiser:  namssi  _ r\  upTflA  .  iflrf - £j«: - — - 

Instructions:  it)  Transfer  tns  OOMAivCAEDr  'OTACS  from  ms  Evaluation  Recorcs  Multiply  Appnaei  2  5y  40  to  06... 


r.  :,ts  Jc'i  -  a.-.jia. 


Domain  mRR 

i  instructional  Strategies 

il  Classroom  Management  ana  Organization 

hi  Presentation  of  Subiect  Maner 

IV  Learning  Environment 

v  Growth  ano  ResponsrDiiiiies 


Domain  Scoras:  v 

(i)  Comoute  me  Domain  Total  by  aaomg  me  aomam  subtotals  ! 
12)  Cons*-:  me  Score  Conversion  Chart  to  convert  each  Domain 


instructional  Strategies 
Classroom  Organization  ano  Managemsnt 
Presentation  of  SuOiect  Maner 
Lea'nmg  Environment 
G'owtn  anc  ResoonsiQiiities 


Appraisal  2 

JJL  . 

-4-  ■ 

dt  : 


Domain  Subtotal 

-O-Jtl 

AC 

40  «  *3  .  lo 


Domain  Score 


Dometn  SuMl 

4k 

JMJL 

10.2. 

JZJD 


in  Subtotal 


Domain  Total 

JJLM 

ZLZ, 

ZLZ, 

1 3.2 

1 2.  A 


Domain  Scora 

A3 


Jk* 


Ovarail  Summary  Parformanca  Scora: 

1 1 1  Enter  me  Domain  Scoras  in  tna  specas  provided  bofow  and  computa  tha  sum. 
12)  0'via#  ma  total  by  5. 


Oomsm  Scoras  for 

Doma-n  i 


Domain  tl 


Domain  III 


Domain 


Domain  V 


AA.  -  M.  .  HA  -  HA  .  £&JZ., 


Average 


i3)  R score  tne  Overall  Summary  Performance  Scora  u  an  ovoraga  of  ma  domain  scores  except  that  the  overall  parformanca  score  can  oe  no  nig ner  man 
<ai  t  9  if  on#  or  more  of  tne  oomem  scoras  ora  i  to  i  . 9  mcfusivs.  or 
to)  2  9  if  two  or  more  of  me  domain  scores  art  2  to  2.9  inclusive  and  no  score  <$  t  to  t  9 

Overall  Summery  Parformanca  Score  ■  LL  *1 


p«cn»r  i  Svoervuor  Sigrww.-»  Cat*  Cc~3i»ieo 


Signature  of  other  appraiser(s)  if  present  at  conference 


i Tne  S'gnavur*  of  me  teecner  noicates  mat  n#  sne  nas  rscsivsa  a  copy  of  tn#  Appraisal  Recoro  ano  ms  results  ot  tns  psrtormsnce  appra  s*  >  ra.i 
see-  c  sc-sseo  M  a  conference  i 


lie*  :  ;-i  .  * 

3' 3*i  " : : — Crr'-i  0"c*  Coc.  '  '  ?.:imo' 
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APPENDIX  B:  ATC  CLASSROOMS  VISITED  BY  THE  CONTRACTOR 
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ATC  Courses  of  Instruction  Observed -by  the  Contractor: 


Lackland  AFB 

•  Recruiter  training  course 

•  Social  actions  course 

•  Security  police  specialist  course 

Keesler  AFB 

•  Air  traffic  control  operator  course 

•  43E  I/O  Radar  maintenance  course 

•  Automatic  radar  tracking  specialist  course 

•  Basic  electronics  course 

Sheppard  AFB 

•  Dental  assistant  specialist  course 

•  Advanced  X-ray  system  course 

•  Diet  therapy  specialist  course 

•  Medical  service  specialist  course 

•  Technical  training  instructor  course 

•  Electrician  specialist  course 

•  Telephone  specialist  course 
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Lowry  AFB 

•  Inventory  management  specialist  course 

•  B52  tail  gun  avionics  course 

•  F 1 5  Aircraft  armament  systems  specialist  course 
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APPENDIX  C 


ATC  INSTRUCTOR  EVALUATION  FORMS 
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INSTRUCTIONS:  Cboefc  (/)  eoeh  ttaa  la  Section  I  ae:  0  •  Odtatuding;  S  •  SaUafactacr,  NI  •  Needs  Improvement.  Items  which  an 
not  appikabi a  to  the  typo  o t  loan*  prawn  tad  win  bo  (N  A).  Ratinp  of  "Needs  Improvement”  tod  "Otfatmuilnf”  vfll  toqoiio  written 
justification  is  Section  L  Ratings  of  "Needs  Impro runout"  will  require  i  follow-up  wjjidofl  to  Soctloo  IV  within  SO  days. _ 


TIMS  ITAKTID 


gFrjrTCTi 


>N  (Last,  rtnt,  Mlddla  Initial) 


SECTION  |. 


A.  PREPARATION 


I  TIMS 


I .  Lmon  plan  currant,  psraonaiizsd,  and  appropriate.  Laaaon  plan 
ngned  by  instructor's  luparviaor. 


|  2.  Classroom  neat  and  orderly.  Stating  arrangement  appropriate. 
Itama  idantifiad  naading  rape k,  nidi  ai  burned  out  lights. 


3.  Training  materials,  aide,  and  equipment  available  prior  to  the 


4.  Nectary  materiait  distributed  rtf  actively. 


8.  CONTENT/STRUCTURE  OF  PRESENTATION 


1.  Objective!  dearly  stated  and  sequence  for  laaon  briefly 
outlined. 


I  2.  All  objacdvaa  covered,  Lesson  flowed  smoothly  from  point 
to  point 


3.  Main  points  reviewed  in  condusion  (or  internal  summary  given  it 
I  lesaen  not  tndedj 


C.  PRESENTATION  SKILLS 


1.  Eye  contact  made  with  students. 


2.  Movement  and  gestures  natural  and  appropriate,  not  forced  or 
distracting. 


3.  Instructor  poisad.  enthusiastic,  tnd  confidant 


0.  COMMUNICATION  SKILLS 


1 .  Correct  inundation  and  grammar  uaad. 


2.  Excessive  usa  of  distracting  mennariama  such  at  "Aid”  and 
"OKs"  not  evident. 


3.  Instructor'!  mica  quality,  volume,  and  variation  (pitch,  rats, 
inflection)  appropriate  for  group  and  damroom  na. 


E.  DEMONSTRATION  SKILLS 


l .  Skills  property  introduced  and  demonstrated. 


2.  Students  involved  in  demonstration,  if  appropriate. 


F.  USE  OF  TRAINING  AIOS/MATERIAIS 


I.  Training  aiddinstructionei  matariaia/equipment  listed  in  POI  uaad 
by  the  mstructor/studtnts  during  lesson. 


2.  Chalkboard  rod  other  visual  aids  usad  in  an  iff  active  manner. 


’»• 
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SECTION  I.  IMMM  ITEMS 


6.  QUESTION/ANSWER  TECHNIQUES 


1.  Qnmtam  phtaaad  daorfy  asd  to  tta  point 


2.  Quaaticna  apprapritta  far  tta  tan 


3.  Vartay  of  quartan  typn  und.  Typaa  of  quartans  uaad  adjurtad 
to  tha  situation. 


4.  Studant  qaariona  ananwrad  adaqimttly. 


H.  MANAGEMENT 


1.  Propar  control  of  ctaa  manta i nod. 


2.  A«propnata  taduuquai  uaad  to  aatat  and  motntm  irodonts. 


3.  Tima  manayd  spprppriatafy.  Laaaon  wall  pacad. 


4.  Inftroctor  proparly  uaad  tta  MIR  instructor. 


S.  Studant  participation  ancouragad. 


f  I-  SUPERVISING  GROUP  ACTIVITY  SKILLS 


1.  Ctar  instructions  prowdod  to  tta  group. 


2.  Group  rnamban  aocourayad  to  parndpata . 


I  MEASUREMENT 


|  1.  Propram  chock  and  patfonimnco/wf  ittan  tata  adminStarad  proparly. 


2.  Propar  critiqua  procaduraa  followad. 


K.  PERSONAL  QUALITIES 


1.  Thorough  knowiadga  of  subject  manor  damonatratsd. 


2.  Inatructor'i  prof aauona linn  tat  tta  propar  axampfa  for  boarlng, 
batawor  and  draaL 


3.  PoaitiM  rapport  with  itudonti  aatablishod. 


L  OTHER 

DHi 

|  1.  Studanta  in  eomplianco  with  appropnata  dram  and  appnrana 
standards 

» 

■■■ 

■ 

1 

2.  Importation  of  mfaty  tmptaaiad  and  eomptama  witb  mfaty 
|  nandardi  anaurad. 

■■■■ 
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STUDENT  CRITIQUE 


INSTRUCTIONS:  flat  thi a  form  to  tvaiuata  training,  atudant  group  or  baaa  support  facilitlta  and  aameaa.  Uaa  a  aapatata  torn  lor 
*ach  arta  doing  a  valuatad;  i.a.,  do  not  conn  ant  on  training  and  data  support  facilitlta  on  tht  i  amt  form. 


COURSE  NO 

COURSE  TITLE 

PERIOO  OP  TRAINING 

1 

A 

- 

PROM 

TO 

NAME  OP  STUOENT  (Optional) 

1 

GRADE 

OATS 

CLASS  NO 

1 

SMi*T 

SQUAOROI 

Cm  Tl  Qu  £  ON  (Chact  ana}  TYP(  OP  CPITIQU1 


^training  !  j  school  so  □  »a*e  support  pacilities  |  j  inqiviqual  □  croup  £^orai _ type 

Cm  Tl  QU  E  RESULTS  REQUESTED  AOORESS/PHOH  E  NUMBER  (For  latm  COB  lac  I,  it  crutqut  raaulla  ma  racuaatad) 

Qyes  □  no 

STUOENT  COMMENTS  (Tail  mOt,  aOai.  onan,  mtiart,  wdr  and  domj  (Conctnua  an  rararaa  It  nacaaaarr) 
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APPENDIX  D 


ARMY  ANO  NAVY  INSTRUCTOR  EVALUATION  FORMS 
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USACMLS  INSTRUCTOR  ASSESSMENT 


Instructor 
L«ims  No. 
Coursa  — — 


Div. 


Dir. 


ITC  Qualified  QTes  □ 


Title. 


Class  Location. 


Data  &  Tine. 


Evaluator . 


ITC  Qualified  0?as  Qno  Assessment  Tots  Qualified  Olaa  Q 


COMPUTATIONS 


Scoring  baaa  600 

Minus  total  nonrated  factors  _  X  6  * 

Tg r  “TET 

Aligned  baaa  score  ^ 


(i)  ^  (f) 


X  100 


*  Round  score  to  neareet  whole  maeberJ 


SECTION 


of  Objoetivaal 


1.  Gains  attention/eotivetes  at  beginning 
of  claaa. 

2.  Identifies  obj active (a)  at  beginning  of 

class. 

3.  Tell  students  application  of  obj  active  (s)  and 
consequence  (a)  of  non-performance  of  tasJcs(s); 
ties  lesson  in  to  pest  instruction. 


•nts:, 


FACTOR 

WRIGHT  X  EVALUATION 


5  Nil  23  45  6 

5  NR  1  2  3  4  5  6 

4  NR  1  2  3  4  5  i 

I*  Subtotal 


EARNED 

POINTS 


SECTION  B.  Wim  Presentation  (Body) 

1.  Lesson  plan  well  structured/current/ IAH 
policy  letter. 

2.  Trains  IAH  lesson  plan. 

3.  Presents  lesson  plan  in  an  interesting  and 
innovative  way  (stories,  huaor.  examples). 

4.  Uses  training  aids  professionally. 

5.  Uses  transitions  and  summaries  effectively. 

6.  Concludes  and  ties  lesson  together  and  uses 
an  appropriate  closing  statement. 

Coanent  s :  -  _ 


6 

8 

6 

5 

4 


NR  1  2  3  4  5  6 
NR  l  2  3  4  5  6 

NR  1  2  3  4  5  6 
NR  1  2  3  4  5  6 
NR  1  2  3  4  5  6 

NR  1  2  3  4  5  6 
Subtotal 


FT  Mc<X  Form  1772-R,  1  Aug  86,  orevioue  editions  obsolete. 
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SECTION  C.  Instructor  Communication  Skills 


FACTOR 

WEIGHT  Z  EVALUATION 


EARNED 

POINTS 


1. 

2. 

3. 

•4. 

5. 

6. 

Comments: 


Phrases  questions  to  elicit  student  thought/ 

correct  answers. 

4 

NR  1 

2  3  4  3  6 

Uses  proper  questioning  techniques. 

4 

NR  1 

2  3  4  5  6 

Speech  &  Voice. 

5 

NR  1 

2  3  4  5  6 

Gestures. 

3 

NR  1 

2  3  4  5  6 

Eye  contact. 

3 

NR  1 

2  3  4  5  6 

Movement. 

NR  1 

2  3  4  5  6 

• 

20 

Subtotal 

SECTION  0.  Instructor  Preparation 


1. 

Civilian  dress  is  test eful/Milit ary  appearance 
is  IAN  AR  670-1  and  local  regulation. 

1 

NR 

2. 

Is  in  control  of  class. 

4 

NR 

3. 

Appears  highly  motivated. 

4 

NR 

4. 

Is  relaxed  and  confident  in  interaction  with 
students. 

3 

NR 

5. 

Shows  concern,  understanding,  and  willingness 
to  help  students. 

3 

NR 

6. 

Exhibits  competence  in  subject  matter. 

6 

NR 

7. 

Ensures  that  doctrine  trained  is  current 
and  correct. 

NR 

27 

NR  1  2  3  4  5  6 


Subtotal 


SECTION  E.  Training  Sits  Management 

2  NR  1  2  3  4  5  6 

1  NR  1  2  3  4  5  6 

2_  NR  1  2  3  4  3  6 

3  Subtotal 


GENERAL  COMMENTS: 


1.  Ensures  that  physical  arrangement  facilitates 
instruction  and  student  involvement. 

2.  Ensures  that  Visitor  Folder  is  IAH  USACMLS 
Policy  Latter  #67. 

3.  Ensures  that  Ala  (when  available)  are 
used  appropriately. 

Comments: 
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(NAVY) 


INSTRUCTIONAL  EFFECTIVENESS  CHECKLIST 

Instructions:  Answer  yes  or  no  to  relevant  questions;  do  not  answer  irrelevant 
questions  nor  force  inappropriate  yes  or  no  answers.  Use  the  back  of  the  form  to 
provide  additional  details  as  required.  Answer  the  following  questions  as  the 
lesson  progresses. 


LEARNING  ORIENTATION 

Yes 

No 

Are  students  given  objectives? 

Clarified/amplified  (if  necessary)? 

Were  the  students  motivated  in  terms  of  "why*  the  content 
should  be  learned? 

Were  the  students  motivated  re  "how”  the  content  should 
be  used? 

Are  students  told  how  they  will  be  tested? 

Ard  students  told  what  they  will  be  tested  on? 

Did  instructor  frequently  orient  students  to 
job  of  learning?  Was  information 
about  the  instructional  process  and  what  the 
students  were  supposed  to  be  doing/learning  given? 
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INSTRUCTOR  BEHAVIOR 


Was  peer  instruction  used? 


Were  external  rewards  given? 


Did  instructor  establish  relationship  w/  Ss  by  introducing 
her/himself? 

by  displaying  course/unit/module  name? 
by  creating  interest  in  subject? 
by  displaying  enthusiasm? 
by  soliciting  class  cooperation  and  involvement? 


Is  the  instructor’s  voice  level  and  enunciation  adequate? 


Was  the  instructor  free  from  any  distracting  mannerisms 
fe.g.ticks,  twitches)? 


Did  it  appear  that  the  instructor  adequately  prepared  for 
the  lesson? 


Did  the  instructor  urge  students  to  take  notes? 


At  appropriate  points  in  lecture,  did  the  instructor 
pause  and  indicate  to  students  they  should  take  notes? 


Did  Instructor  allow  students  to  learn  by 
doing  when  possible? 


Did  instructor  insure  that  students  were  actively  involved? 


Did  instructor  monitor  student  progress? 


Did  instructor  provide  assistance  when  necess 
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MEDIA  /  MATERIALS 


What  types  of  media  were  used? _ - _ 


Were  they  appropriate? 


Were  the  transparencies, etc,  technically  correct 
to  read,  layout  attractive,  etc. 


Were  there  sufficient  number  of  instructional  aids? 
(example,  could  transparencies  be  used  instead 
of  blackboard? 


Were  they  effectively  used  in  separating  the  various 
instructional  activities/phases/summ 


ENVIRONMENT  /  SAFETY 


Describe  significant  environmental  factors  involving 
light,  background  noise,  temperature,  general 
attractiveness  of  classroom,  etc. 


Was  the  equipment  operational? 


Were  necessary  tools  and  test  equipment  available? 


Was  lab  clean  and  free  of  safety  hazards? 


Was  two  man  rule  observed? 


recautions  announced? 


Were  students  aware  of  emergency  procedures? 


Was  emergency  first  aid  procedure  posted 
and  visible? 


Were  high  voltage  areas  clearly  marked? 


Yes  No 
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STUDENT  BEHAVIOR 

Yes 

No 

Did  it  appear  the  students  achieved  the  objectives? 

Did  it  appear  the  student  took  adequate  amount  of 
notes? 

How  involved  were  the  students  in  the  learning 
process?  Circle  the  most  appropriate  category 
listed  below  that  describes  the  majority  of 
the  students. 

ACTIVE  Student  asks  questions,  answers 

INVOLVEMENT  questions,  volunteers  extra  information 
generates  information  him /herself.  May 
go  beyond  information  immediately  given 
and  relate  to  other  known  information. 

ATTENTIVE  Student  clearly  understands  and  is  tracking 
INVOLVEMENT  what  teacher  is  saying.  Seems  interested. 
Indicated  by  alert  faces,  heads  nodding 
positively,  alert  posture,  smiling,  talking 
to  self. 

PASSIVE  Student  seems  bored  and  is  only  marginally 

INVOLVEMENT  paying  attention  to  what  is  going  on  in  the 
class.  Is  able  to  answer  straighforward 
questions,  but  does  not  exhibit  a  great 
deal  of  interest  ;n  what  is  being  taught. 

NO  Students’  eyes  may  be  glazed  over  and  may 

INVOLVEMENT  have  difficulty  staying  awake.  Can't  answer 
question  if  called  upon.  May  not 
even  seem  to  know  what  lesson  instructor 
is  addressing. 

no 


